[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:34 dist-test-slave-1x4g krb5kdc[15143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:34 dist-test-slave-1x4g krb5kdc[15143](info): set up 2 sockets
Aug 10 20:59:34 dist-test-slave-1x4g krb5kdc[15143](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 20:59:35.403156 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:35 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:35.522969 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43235
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:35.608927 15158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:35.609141 15158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:35.609179 15158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:35.610342 15158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:35.610407 15158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:35.610431 15158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:35.610458 15158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:35.610481 15158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:35.612177 15158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:35.612567 15158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:35.613045 15158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:35.620138 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:35.620474 15163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:35.620712 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:35.622014 15158 server_base.cc:1034] running on GCE node
I20260810 20:59:35.622290 15158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:35.622588 15158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:35.623776 15158 hybrid_clock.cc:648] HybridClock initialized: now 1786395575623739 us; error 54 us; skew 500 ppm
Aug 10 20:59:35 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:35.630484 15158 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:35.631119 15158 webserver.cc:460] Webserver started at http://127.14.200.62:42825/ using document root <none> and password file <none>
I20260810 20:59:35.631366 15158 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:35.631407 15158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:35.631552 15158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:35.632848 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "fa12544a4ded4ed5ae962d8332c14b11"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "78c895950d54a6fa4e794a858bd86142"
server_key_iv: "36987f90212429c55b1b3d06f8557493"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.633507 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "fa12544a4ded4ed5ae962d8332c14b11"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "78c895950d54a6fa4e794a858bd86142"
server_key_iv: "36987f90212429c55b1b3d06f8557493"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.635326 15158 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 20:59:35.636229 15173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:35.636448 15158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:35.636523 15158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "fa12544a4ded4ed5ae962d8332c14b11"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "78c895950d54a6fa4e794a858bd86142"
server_key_iv: "36987f90212429c55b1b3d06f8557493"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.636608 15158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:35.652380 15158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:35.654922 15158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:35.655086 15158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:35.666134 15158 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:43235
I20260810 20:59:35.666282 15236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:43235 every 8 connection(s)
I20260810 20:59:35.666708 15158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:35.667742 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15158
I20260810 20:59:35.667845 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:35.668048 15237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:35.668533 15136 external_mini_cluster.cc:1428] Setting key 52e2bfbf277e8cd0645360afa1f24b68
I20260810 20:59:35.671296 15237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11: Bootstrap starting.
I20260810 20:59:35.672117 15237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:35.672451 15237 log.cc:824] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:35.673564 15237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11: No bootstrap required, opened a new log
I20260810 20:59:35.678002 15237 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } }
I20260810 20:59:35.678193 15237 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:35.678275 15237 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa12544a4ded4ed5ae962d8332c14b11, State: Initialized, Role: FOLLOWER
I20260810 20:59:35.678465 15237 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } }
I20260810 20:59:35.678580 15237 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:35.678603 15237 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:35.678670 15237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:35.679502 15237 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } }
I20260810 20:59:35.679734 15237 leader_election.cc:302] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa12544a4ded4ed5ae962d8332c14b11; no voters:
I20260810 20:59:35.679975 15237 leader_election.cc:288] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:35.680084 15243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:35.680384 15243 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [term 1 LEADER]: Becoming Leader. State: Replica: fa12544a4ded4ed5ae962d8332c14b11, State: Running, Role: LEADER
I20260810 20:59:35.680496 15237 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:35.680534 15243 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } }
I20260810 20:59:35.681200 15245 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa12544a4ded4ed5ae962d8332c14b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } } }
I20260810 20:59:35.681294 15245 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:35.681320 15243 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa12544a4ded4ed5ae962d8332c14b11. Latest consensus state: current_term: 1 leader_uuid: "fa12544a4ded4ed5ae962d8332c14b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa12544a4ded4ed5ae962d8332c14b11" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43235 } } }
I20260810 20:59:35.681392 15243 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:35.681936 15254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:35.683368 15254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 20:59:35 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:35.685913 15254 catalog_manager.cc:1347] Generated new cluster ID: 8693f934eaea4570869b5db55ed8e4b3
I20260810 20:59:35.685968 15254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:35.690781 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:35.669585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.14.200.62:43235)
0810 20:59:35.670065 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:35.670083 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:35.670309 (+ 226us) server_negotiation.cc:407] Connection header received
0810 20:59:35.673438 (+ 3129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:35.673447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:35.673570 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:35.673748 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:35.675500 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.677761 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.679213 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.679979 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.680194 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:35.686181 (+ 5987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:35.686185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:35.686185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:35.686203 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:35.688521 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:35.689116 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:35.689119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:35.689119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:35.689222 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:35.689568 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:35.689568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:35.689569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:35.689837 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:35.690064 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:35.690342 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 20:59:35.690487 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":180,"threads_started":1}
I20260810 20:59:35.694370 15254 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:35.694875 15254 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:35.706856 15254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa12544a4ded4ed5ae962d8332c14b11: Generated new TSK 0
I20260810 20:59:35.707026 15254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:35.739974 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:35.861485 15268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:35.861675 15268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:35.861698 15268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:35.862809 15268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:35.862851 15268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:35.862879 15268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:35.864346 15268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:35.864639 15268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:35.864943 15268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:35.870965 15276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:35.870926 15273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:35.870942 15274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:35.871315 15268 server_base.cc:1034] running on GCE node
I20260810 20:59:35.871484 15268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:35.871688 15268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:35.872853 15268 hybrid_clock.cc:648] HybridClock initialized: now 1786395575872833 us; error 22 us; skew 500 ppm
Aug 10 20:59:35 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:35.875262 15268 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:35.875775 15268 webserver.cc:460] Webserver started at http://127.14.200.1:38889/ using document root <none> and password file <none>
I20260810 20:59:35.876014 15268 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:35.876081 15268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:35.876214 15268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:35.877450 15268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "99f51cea439c40a4a2d132229364eb49"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "a303c0ebca3b0c44d3aafbf0c4f02295"
server_key_iv: "6b29b1848cfea93430d392f59d1cd899"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.877815 15268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "99f51cea439c40a4a2d132229364eb49"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "a303c0ebca3b0c44d3aafbf0c4f02295"
server_key_iv: "6b29b1848cfea93430d392f59d1cd899"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.879101 15268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:35.879781 15285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:35.879980 15268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:35.880074 15268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "99f51cea439c40a4a2d132229364eb49"
format_stamp: "Formatted at 2026-08-10 20:59:35 on dist-test-slave-1x4g"
server_key: "a303c0ebca3b0c44d3aafbf0c4f02295"
server_key_iv: "6b29b1848cfea93430d392f59d1cd899"
server_key_version: "encryptionkey@0"
I20260810 20:59:35.880151 15268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:35.896065 15268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:35.897812 15268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:35.898001 15268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:35.898262 15268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:35.898612 15268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:35.898703 15268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:35.898743 15268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:35.898769 15268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:35.906562 15268 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:45795
I20260810 20:59:35.906710 15410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:45795 every 8 connection(s)
I20260810 20:59:35.906967 15268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 20:59:35 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395575, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:35.916168 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15268
I20260810 20:59:35.916287 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:35.916447 15136 external_mini_cluster.cc:1428] Setting key 8929eac1e011266ef980d1daeeda08bf
I20260810 20:59:35.916992 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:35.907821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:35397 (local address 127.14.200.62:43235)
0810 20:59:35.907867 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:35.907869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:35.908133 (+ 264us) server_negotiation.cc:407] Connection header received
0810 20:59:35.908791 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:35.908792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:35.908810 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:35.908863 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:35.909974 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.910470 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.911527 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.911784 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.911869 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:35.914250 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:35.914252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:35.914252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:35.914259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:35.915607 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:35.915940 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:35.915941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:35.915941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:35.915993 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:35.916313 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:35.916314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:35.916314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:35.916458 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:35.916531 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:35.916859 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 20:59:35.916928 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 20:59:35.917230 15413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:35.907768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43235 (local address 127.14.200.1:35397)
0810 20:59:35.908014 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:35.908048 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:35.908663 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:35.908942 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:35.908946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:35.909607 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:35.909860 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:35.909865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.910572 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.910573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:35.911423 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:35.911426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:35.911882 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:35.911883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:35.912080 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:35.912602 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:35.912610 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:35.914157 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:35.915675 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:35.915680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:35.915682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:35.915876 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:35.916064 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:35.916065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:35.916066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:35.916213 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:35.916537 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:35.916545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:35.916676 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 20:59:35.916824 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 20:59:35.916992 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":167,"threads_started":1}
I20260810 20:59:35.917806 15411 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43235
I20260810 20:59:35.917922 15411 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:35.918099 15411 heartbeater.cc:511] Master 127.14.200.62:43235 requested a full tablet report, sending...
I20260810 20:59:35.918911 15193 ts_manager.cc:194] Registered new tserver with Master: 99f51cea439c40a4a2d132229364eb49 (127.14.200.1:45795)
I20260810 20:59:35.919642 15193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:35397
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:35.943872 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:36.058250 15417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:36.058434 15417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:36.058458 15417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:36.059621 15417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:36.059671 15417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:36.059696 15417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:36.061183 15417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:36.061355 15417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:36.061544 15417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:36.067701 15426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:36.067729 15424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:36.067974 15417 server_base.cc:1034] running on GCE node
W20260810 20:59:36.067895 15423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:36.068251 15417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:36.068497 15417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:36.069650 15417 hybrid_clock.cc:648] HybridClock initialized: now 1786395576069640 us; error 41 us; skew 500 ppm
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:36.072372 15417 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:36.072853 15417 webserver.cc:460] Webserver started at http://127.14.200.2:46645/ using document root <none> and password file <none>
I20260810 20:59:36.073052 15417 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:36.073091 15417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:36.073179 15417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:36.074172 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "b3e9f1f8795d4932a6cee71de79a6686"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "4a0b270d8cb5536826d6bde966c4d90a"
server_key_iv: "7f9faac6978e056e1b609f3c46880b3f"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.074497 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "b3e9f1f8795d4932a6cee71de79a6686"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "4a0b270d8cb5536826d6bde966c4d90a"
server_key_iv: "7f9faac6978e056e1b609f3c46880b3f"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.075928 15417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:36.076804 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.076969 15417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:36.077060 15417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "b3e9f1f8795d4932a6cee71de79a6686"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "4a0b270d8cb5536826d6bde966c4d90a"
server_key_iv: "7f9faac6978e056e1b609f3c46880b3f"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.077114 15417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:36.089246 15417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:36.091132 15417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:36.091317 15417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:36.091760 15417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:36.092149 15417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:36.092195 15417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.092221 15417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:36.092235 15417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.106101 15417 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46111
I20260810 20:59:36.106137 15561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46111 every 8 connection(s)
I20260810 20:59:36.106537 15417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 20:59:36.109503 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15417
I20260810 20:59:36.109607 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:36.109812 15136 external_mini_cluster.cc:1428] Setting key 60210d27a69f79420cfc97c34ceef320
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:36.117738 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.107378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:33717 (local address 127.14.200.62:43235)
0810 20:59:36.107507 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:36.107510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:36.107821 (+ 311us) server_negotiation.cc:407] Connection header received
0810 20:59:36.108473 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:36.108474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:36.108484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:36.108530 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:36.109737 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.110369 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.111983 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.112164 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.112217 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.114888 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:36.114890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:36.114890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:36.114897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:36.116292 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.116749 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.116751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.116751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.116837 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.117085 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.117086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.117086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.117207 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:36.117280 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:36.117559 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 20:59:36.117651 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 20:59:36.117930 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.107332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43235 (local address 127.14.200.2:33717)
0810 20:59:36.107683 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:36.107719 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:36.108383 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:36.108588 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:36.108591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:36.109203 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:36.109596 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:36.109604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.110443 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.110444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:36.111899 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:36.111902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.112246 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.112247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:36.112478 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.113310 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:36.113318 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:36.114791 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:36.116389 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:36.116396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:36.116398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:36.116622 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:36.116903 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:36.116904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:36.116905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:36.117010 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:36.117284 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:36.117291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:36.117418 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 20:59:36.117534 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 20:59:36.117689 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":224,"threads_started":1}
I20260810 20:59:36.118453 15562 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43235
I20260810 20:59:36.118536 15562 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:36.118731 15562 heartbeater.cc:511] Master 127.14.200.62:43235 requested a full tablet report, sending...
I20260810 20:59:36.119124 15193 ts_manager.cc:194] Registered new tserver with Master: b3e9f1f8795d4932a6cee71de79a6686 (127.14.200.2:46111)
I20260810 20:59:36.119705 15193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:33717
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:36.137539 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:36.252671 15568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:36.252975 15568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:36.253003 15568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:36.254218 15568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:36.254256 15568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:36.254304 15568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:36.255837 15568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:36.256078 15568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:36.256300 15568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:36.262456 15575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:36.262513 15576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:36.262459 15579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:36.262705 15568 server_base.cc:1034] running on GCE node
I20260810 20:59:36.262933 15568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:36.263199 15568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:36.264350 15568 hybrid_clock.cc:648] HybridClock initialized: now 1786395576264341 us; error 26 us; skew 500 ppm
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:36.267653 15568 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:36.268102 15568 webserver.cc:460] Webserver started at http://127.14.200.3:38057/ using document root <none> and password file <none>
I20260810 20:59:36.268323 15568 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:36.268389 15568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:36.268541 15568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:36.269615 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "7acb31a2c09a44dcb64d4219fa8574a5"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "fb3b6890391d11dabbe224686e5c03f8"
server_key_iv: "839824ea820c27de9b6dcd1da92baa8c"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.270043 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "7acb31a2c09a44dcb64d4219fa8574a5"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "fb3b6890391d11dabbe224686e5c03f8"
server_key_iv: "839824ea820c27de9b6dcd1da92baa8c"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.271651 15568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:36.272485 15586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.272693 15568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:36.272841 15568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "7acb31a2c09a44dcb64d4219fa8574a5"
format_stamp: "Formatted at 2026-08-10 20:59:36 on dist-test-slave-1x4g"
server_key: "fb3b6890391d11dabbe224686e5c03f8"
server_key_iv: "839824ea820c27de9b6dcd1da92baa8c"
server_key_version: "encryptionkey@0"
I20260810 20:59:36.272915 15568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:36.293404 15568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:36.295476 15568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:36.295683 15568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:36.296088 15568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:36.296449 15568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:36.296504 15568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.296536 15568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:36.296583 15568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:36.311964 15568 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:35973
I20260810 20:59:36.312124 15717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:35973 every 8 connection(s)
I20260810 20:59:36.312425 15568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:36.313549 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15568
I20260810 20:59:36.313642 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:36.313834 15136 external_mini_cluster.cc:1428] Setting key d11142ba13373bf091c80e42447629d2
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:36.322902 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.313301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:40227 (local address 127.14.200.62:43235)
0810 20:59:36.313377 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:36.313379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:36.313834 (+ 455us) server_negotiation.cc:407] Connection header received
0810 20:59:36.314889 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:36.314890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:36.314903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:36.315002 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:36.316159 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.316689 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.317754 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.317952 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.318003 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.320493 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:36.320494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:36.320494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:36.320501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:36.321766 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.322121 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.322122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.322122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.322170 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.322371 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.322372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.322372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.322468 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:36.322516 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:36.322753 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 20:59:36.322810 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 20:59:36.323078 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.313286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43235 (local address 127.14.200.3:40227)
0810 20:59:36.313675 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:36.313718 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:36.314759 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:36.315082 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:36.315087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:36.315758 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:36.316076 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:36.316082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.316769 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.316770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:36.317683 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:36.317685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.318030 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.318031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:36.318179 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.318749 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:36.318757 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:36.320399 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:36.321837 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:36.321843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:36.321845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:36.322059 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:36.322226 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:36.322227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:36.322227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:36.322319 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:36.322522 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:36.322526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:36.322628 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 20:59:36.322725 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 20:59:36.322838 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":250,"threads_started":1}
I20260810 20:59:36.323661 15719 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43235
I20260810 20:59:36.323773 15719 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:36.323947 15719 heartbeater.cc:511] Master 127.14.200.62:43235 requested a full tablet report, sending...
I20260810 20:59:36.324329 15193 ts_manager.cc:194] Registered new tserver with Master: 7acb31a2c09a44dcb64d4219fa8574a5 (127.14.200.3:35973)
I20260810 20:59:36.324692 15193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:40227
I20260810 20:59:36.326175 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:36.351357 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.343167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.14.200.1:45795)
0810 20:59:36.343469 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:36.343473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:36.343483 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:36.343508 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:36.343510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:36.343523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:36.343605 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:36.344229 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.345171 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.345971 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.346357 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.346415 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.348717 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:36.348724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:36.348724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:36.348730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:36.350307 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.350607 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.350609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.350610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.350733 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.350897 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.350898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.350898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.351026 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:36.351120 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:36.351182 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 20:59:36.351242 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":127,"threads_started":1}
W20260810 20:59:36.351890 15313 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39716
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:36.375955 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:36.369343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39720 (local address 127.14.200.1:45795)
0810 20:59:36.369385 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:36.369386 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:36.369478 (+ 92us) server_negotiation.cc:407] Connection header received
0810 20:59:36.369533 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:36.369534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:36.369542 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:36.369629 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:36.370124 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.370613 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.371270 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:36.371499 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:36.371552 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:36.373566 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:36.373567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:36.373567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:36.373574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:36.374995 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.375271 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.375272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.375272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.375323 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:36.375464 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:36.375464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:36.375464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:36.375578 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:36.375708 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:36.375737 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 20:59:36.375829 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 20:59:36.376488 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15268
I20260810 20:59:36.384038 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15417
I20260810 20:59:36.390156 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15568
I20260810 20:59:36.396145 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15158
2026-08-10T20:59:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2087 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15734](info): set up 2 sockets
Aug 10 20:59:36 dist-test-slave-1x4g krb5kdc[15734](info): commencing operation
krb5kdc: starting...
W20260810 20:59:37.445417 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:37.559203 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45529
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:37.642901 15747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:37.643103 15747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:37.643127 15747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:37.644266 15747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:37.644315 15747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:37.644332 15747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:37.644389 15747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:37.644415 15747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:37.646562 15747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:37.646821 15747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:37.647080 15747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:37.652055 15752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.652072 15755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.652065 15753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:37.652227 15747 server_base.cc:1034] running on GCE node
I20260810 20:59:37.652570 15747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:37.652935 15747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:37.654080 15747 hybrid_clock.cc:648] HybridClock initialized: now 1786395577654061 us; error 44 us; skew 500 ppm
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:37.657039 15747 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:37.657447 15747 webserver.cc:460] Webserver started at http://127.14.200.62:36901/ using document root <none> and password file <none>
I20260810 20:59:37.657670 15747 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:37.657713 15747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:37.657857 15747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:37.658895 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "89bfe8767c3a4bf58b338780a2cd2b76"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "ac1e2322eb3918fdc14345b1cd84c32d"
server_key_iv: "79ada50101b7255b2b801936a41ef46e"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.659274 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "89bfe8767c3a4bf58b338780a2cd2b76"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "ac1e2322eb3918fdc14345b1cd84c32d"
server_key_iv: "79ada50101b7255b2b801936a41ef46e"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.660609 15747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:37.661415 15763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.661594 15747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.661703 15747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "89bfe8767c3a4bf58b338780a2cd2b76"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "ac1e2322eb3918fdc14345b1cd84c32d"
server_key_iv: "79ada50101b7255b2b801936a41ef46e"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.661775 15747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:37.678639 15747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:37.680317 15747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:37.680472 15747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:37.684543 15747 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45529
I20260810 20:59:37.684594 15824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45529 every 8 connection(s)
I20260810 20:59:37.685001 15747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:37.685883 15825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:37.688091 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: Bootstrap starting.
I20260810 20:59:37.688663 15825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:37.688982 15825 log.cc:824] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:37.689647 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: No bootstrap required, opened a new log
I20260810 20:59:37.691205 15825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } }
I20260810 20:59:37.691345 15825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:37.691392 15825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bfe8767c3a4bf58b338780a2cd2b76, State: Initialized, Role: FOLLOWER
I20260810 20:59:37.691537 15825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } }
I20260810 20:59:37.691598 15825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:37.691618 15825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:37.691682 15825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:37.692178 15825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } }
I20260810 20:59:37.692286 15825 leader_election.cc:302] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89bfe8767c3a4bf58b338780a2cd2b76; no voters:
I20260810 20:59:37.692457 15825 leader_election.cc:288] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:37.692510 15829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:37.692679 15829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [term 1 LEADER]: Becoming Leader. State: Replica: 89bfe8767c3a4bf58b338780a2cd2b76, State: Running, Role: LEADER
I20260810 20:59:37.692723 15825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:37.692847 15829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } }
I20260810 20:59:37.693262 15831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89bfe8767c3a4bf58b338780a2cd2b76. Latest consensus state: current_term: 1 leader_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } } }
I20260810 20:59:37.693346 15831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:37.693493 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bfe8767c3a4bf58b338780a2cd2b76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45529 } } }
I20260810 20:59:37.693553 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:37.693796 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15747
I20260810 20:59:37.693886 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:37.694034 15136 external_mini_cluster.cc:1428] Setting key 86340908c11332d7eb696f9be7aee907
W20260810 20:59:37.694599 15846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 20:59:37.694696 15846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 20:59:37.694819 15839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:37.694998 15839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:37.696305 15839 catalog_manager.cc:1347] Generated new cluster ID: a4197a4df51b4fa2ba4deabd1fa02ad0
I20260810 20:59:37.696354 15839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:37.705211 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:37.694589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.14.200.62:45529)
0810 20:59:37.694756 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:37.694783 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:37.694817 (+ 34us) server_negotiation.cc:407] Connection header received
0810 20:59:37.694928 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:37.694946 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:37.695000 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:37.695133 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:37.695784 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.697226 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.698317 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.698914 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.699012 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:37.701613 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:37.701617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:37.701617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:37.701629 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:37.703549 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:37.703926 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:37.703929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:37.703930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:37.704061 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:37.704262 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:37.704263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:37.704263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:37.704585 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:37.704758 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:37.704851 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 20:59:37.704985 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260810 20:59:37.712365 15839 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:37.712997 15839 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:37.716113 15839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89bfe8767c3a4bf58b338780a2cd2b76: Generated new TSK 0
I20260810 20:59:37.716284 15839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:37.743784 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:37.823675 15853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:37.823868 15853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:37.823890 15853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:37.824988 15853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:37.825022 15853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:37.825047 15853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:37.826378 15853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:37.826545 15853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:37.826710 15853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:37.831609 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.831645 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.831617 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:37.831681 15853 server_base.cc:1034] running on GCE node
I20260810 20:59:37.832057 15853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:37.832331 15853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:37.833469 15853 hybrid_clock.cc:648] HybridClock initialized: now 1786395577833457 us; error 30 us; skew 500 ppm
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:37.835980 15853 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:37.836460 15853 webserver.cc:460] Webserver started at http://127.14.200.1:35283/ using document root <none> and password file <none>
I20260810 20:59:37.836678 15853 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:37.836724 15853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:37.836889 15853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:37.837966 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "bf77524af7fb4bbda9d78febd1fa368e"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "3bc6f740dbe4260e749450b8987351a1"
server_key_iv: "be9001a0d367a504a3c0dc63915fb203"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.838284 15853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "bf77524af7fb4bbda9d78febd1fa368e"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "3bc6f740dbe4260e749450b8987351a1"
server_key_iv: "be9001a0d367a504a3c0dc63915fb203"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.839550 15853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:37.840270 15869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.840449 15853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:37.840533 15853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "bf77524af7fb4bbda9d78febd1fa368e"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "3bc6f740dbe4260e749450b8987351a1"
server_key_iv: "be9001a0d367a504a3c0dc63915fb203"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.840585 15853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:37.847033 15853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:37.848904 15853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:37.849010 15853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:37.849165 15853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:37.849479 15853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:37.849519 15853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.849562 15853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:37.849587 15853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.855028 15853 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:39595
I20260810 20:59:37.855099 15999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:39595 every 8 connection(s)
I20260810 20:59:37.855392 15853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 20:59:37.858250 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 15853
I20260810 20:59:37.858347 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:37.858501 15136 external_mini_cluster.cc:1428] Setting key 11ecdd6af1ce0c245ebe7a92b2597b8b
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:37.865525 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:37.856105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53525 (local address 127.14.200.62:45529)
0810 20:59:37.856166 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:37.856170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:37.856363 (+ 193us) server_negotiation.cc:407] Connection header received
0810 20:59:37.857497 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:37.857498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:37.857514 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:37.857567 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:37.858230 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.858896 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.859901 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.860121 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.860206 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:37.862461 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:37.862463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:37.862463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:37.862470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:37.864129 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:37.864492 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:37.864494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:37.864494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:37.864567 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:37.864838 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:37.864839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:37.864839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:37.864994 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:37.865063 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:37.865334 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 20:59:37.865424 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 20:59:37.865705 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:37.856091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45529 (local address 127.14.200.1:53525)
0810 20:59:37.856254 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:37.856286 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:37.857372 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:37.857645 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:37.857648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:37.857935 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:37.858128 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:37.858132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.859005 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.859006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:37.859813 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:37.859815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:37.860221 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:37.860222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:37.860335 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:37.860864 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:37.860873 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:37.862366 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:37.864202 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:37.864209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:37.864210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:37.864411 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:37.864642 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:37.864642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:37.864643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:37.864740 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:37.865068 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:37.865074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:37.865182 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 20:59:37.865288 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 20:59:37.865440 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260810 20:59:37.866190 16000 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45529
I20260810 20:59:37.866295 16000 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:37.866474 16000 heartbeater.cc:511] Master 127.14.200.62:45529 requested a full tablet report, sending...
I20260810 20:59:37.866958 15783 ts_manager.cc:194] Registered new tserver with Master: bf77524af7fb4bbda9d78febd1fa368e (127.14.200.1:39595)
I20260810 20:59:37.867539 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:53525
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:37.883704 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:37.962412 16005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:37.962586 16005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:37.962608 16005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:37.963778 16005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:37.963814 16005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:37.963840 16005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:37.965396 16005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:37.965633 16005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:37.965844 16005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:37.970484 16015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.970486 16011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:37.970562 16012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:37.970666 16005 server_base.cc:1034] running on GCE node
I20260810 20:59:37.970955 16005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:37.971213 16005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:37.972373 16005 hybrid_clock.cc:648] HybridClock initialized: now 1786395577972362 us; error 37 us; skew 500 ppm
Aug 10 20:59:37 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:37.974846 16005 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:37.975178 16005 webserver.cc:460] Webserver started at http://127.14.200.2:44267/ using document root <none> and password file <none>
I20260810 20:59:37.975350 16005 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:37.975386 16005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:37.975467 16005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:37.976310 16005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "6975dd5cd89f4a03a211e9500cf06445"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "4e5e71db7e63833b1aedf56da526d1e4"
server_key_iv: "2a7eadb42637fab686d4daee1bc9c001"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.976629 16005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "6975dd5cd89f4a03a211e9500cf06445"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "4e5e71db7e63833b1aedf56da526d1e4"
server_key_iv: "2a7eadb42637fab686d4daee1bc9c001"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.977973 16005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:37.978681 16023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:37.978844 16005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:37.978920 16005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "6975dd5cd89f4a03a211e9500cf06445"
format_stamp: "Formatted at 2026-08-10 20:59:37 on dist-test-slave-1x4g"
server_key: "4e5e71db7e63833b1aedf56da526d1e4"
server_key_iv: "2a7eadb42637fab686d4daee1bc9c001"
server_key_version: "encryptionkey@0"
I20260810 20:59:37.978971 16005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:37.997905 16005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:37.999876 16005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:37.999975 16005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:38.000126 16005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:38.000407 16005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:38.000447 16005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:38.000466 16005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:38.000514 16005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:38.005986 16005 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46329
I20260810 20:59:38.006073 16152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46329 every 8 connection(s)
I20260810 20:59:38.006358 16005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 20:59:38.008414 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16005
I20260810 20:59:38.008490 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:38.008648 15136 external_mini_cluster.cc:1428] Setting key 64745bf15449a91130c7df478f0cfbce
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:38.016238 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.007064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54171 (local address 127.14.200.62:45529)
0810 20:59:38.007165 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.007168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.007299 (+ 131us) server_negotiation.cc:407] Connection header received
0810 20:59:38.008180 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.008181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.008191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.008240 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.009001 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.009720 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.010717 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.011047 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.011138 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.013472 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.013473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.013474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.013481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.014957 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.015339 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.015340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.015341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.015399 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.015644 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.015645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.015645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.015748 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.015823 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.016078 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.016155 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 20:59:38.016393 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.007001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45529 (local address 127.14.200.2:54171)
0810 20:59:38.007154 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.007194 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.008090 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.008305 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.008308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.008618 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:38.008914 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.008922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.009829 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.009830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.010631 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.010633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.011148 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.011149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.011314 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.011764 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:38.011774 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:38.013354 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:38.015054 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:38.015060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:38.015062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:38.015267 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:38.015466 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:38.015467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:38.015467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:38.015573 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:38.015815 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:38.015822 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:38.015951 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.016046 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.016185 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":90,"threads_started":1}
I20260810 20:59:38.016973 16153 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45529
I20260810 20:59:38.017056 16153 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:38.017249 16153 heartbeater.cc:511] Master 127.14.200.62:45529 requested a full tablet report, sending...
I20260810 20:59:38.017632 15783 ts_manager.cc:194] Registered new tserver with Master: 6975dd5cd89f4a03a211e9500cf06445 (127.14.200.2:46329)
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
I20260810 20:59:38.018121 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:54171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:38.035903 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:38.115182 16158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:38.115362 16158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:38.115384 16158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:38.116569 16158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:38.116667 16158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:38.116717 16158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:38.118244 16158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:38.118497 16158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:38.118713 16158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:38.123499 16165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:38.123499 16168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:38.123520 16166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:38.123821 16158 server_base.cc:1034] running on GCE node
I20260810 20:59:38.124032 16158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:38.124264 16158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:38.125394 16158 hybrid_clock.cc:648] HybridClock initialized: now 1786395578125378 us; error 35 us; skew 500 ppm
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:38.127892 16158 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:38.128226 16158 webserver.cc:460] Webserver started at http://127.14.200.3:37893/ using document root <none> and password file <none>
I20260810 20:59:38.128381 16158 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:38.128417 16158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:38.128504 16158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:38.129498 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "ad5179458724436bafacf676b63ac360"
format_stamp: "Formatted at 2026-08-10 20:59:38 on dist-test-slave-1x4g"
server_key: "5f60a105b3198a83270b5eacfc2597ee"
server_key_iv: "fec44979e7bd3e9176d07f9f817f1e88"
server_key_version: "encryptionkey@0"
I20260810 20:59:38.129876 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "ad5179458724436bafacf676b63ac360"
format_stamp: "Formatted at 2026-08-10 20:59:38 on dist-test-slave-1x4g"
server_key: "5f60a105b3198a83270b5eacfc2597ee"
server_key_iv: "fec44979e7bd3e9176d07f9f817f1e88"
server_key_version: "encryptionkey@0"
I20260810 20:59:38.131103 16158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:38.131772 16175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:38.131939 16158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:38.132045 16158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ad5179458724436bafacf676b63ac360"
format_stamp: "Formatted at 2026-08-10 20:59:38 on dist-test-slave-1x4g"
server_key: "5f60a105b3198a83270b5eacfc2597ee"
server_key_iv: "fec44979e7bd3e9176d07f9f817f1e88"
server_key_version: "encryptionkey@0"
I20260810 20:59:38.132119 16158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:38.142881 16158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:38.145251 16158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:38.145366 16158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:38.145560 16158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:38.145810 16158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:38.145840 16158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:38.145861 16158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:38.145874 16158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:38.152096 16158 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:46239
I20260810 20:59:38.152166 16301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:46239 every 8 connection(s)
I20260810 20:59:38.152491 16158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:38.160254 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16158
I20260810 20:59:38.160331 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:38.160512 15136 external_mini_cluster.cc:1428] Setting key 754a8b2f9933a0a90d217486d60fbdc4
I20260810 20:59:38.161607 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.153303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:50261 (local address 127.14.200.62:45529)
0810 20:59:38.153385 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.153388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.153556 (+ 168us) server_negotiation.cc:407] Connection header received
0810 20:59:38.154473 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.154474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.154485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.154526 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.155074 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.155549 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.156460 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.156666 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.156715 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.158711 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.158712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.158712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.158718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.160052 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.160466 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.160468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.160468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.160529 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.160900 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.160901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.160902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.161048 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.161130 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.161446 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.161522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 20:59:38.161782 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.153249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45529 (local address 127.14.200.3:50261)
0810 20:59:38.153434 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.153470 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.154368 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.154584 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.154587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.154826 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:38.155006 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.155010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.155625 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.155626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.156384 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.156386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.156745 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.156746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.156906 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.157319 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:38.157327 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:38.158630 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:38.160145 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:38.160150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:38.160152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:38.160404 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:38.160603 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:38.160604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:38.160605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:38.160828 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:38.161116 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:38.161123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:38.161286 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.161418 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.161573 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260810 20:59:38.162159 16302 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45529
I20260810 20:59:38.162249 16302 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:38.162405 16302 heartbeater.cc:511] Master 127.14.200.62:45529 requested a full tablet report, sending...
I20260810 20:59:38.162765 15783 ts_manager.cc:194] Registered new tserver with Master: ad5179458724436bafacf676b63ac360 (127.14.200.3:46239)
I20260810 20:59:38.163143 15783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:50261
I20260810 20:59:38.172180 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 20:59:38.179066 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.173997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.14.200.62:45529)
0810 20:59:38.174071 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.174073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.174150 (+ 77us) server_negotiation.cc:407] Connection header received
0810 20:59:38.174220 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.174221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.174228 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.174262 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.174759 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.175171 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.175771 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.175924 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.175969 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.176990 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.176991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.176991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.176997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.178261 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.178521 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.178521 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.178521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.178567 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.178699 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.178699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.178700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.178800 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.178862 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.178941 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.178992 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 20:59:38.181712 15783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:38.183585 15783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:38.190721 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.14.200.3:46239)
0810 20:59:38.186872 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.186876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.186900 (+ 24us) server_negotiation.cc:407] Connection header received
0810 20:59:38.187023 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.187026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.187041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.187097 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:38.187415 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188376 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.189491 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190183 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190258 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190312 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.190503 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.190559 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":107,"threads_started":1}
I20260810 20:59:38.190733 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.14.200.2:46329)
0810 20:59:38.186839 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.186843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.186853 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:38.186965 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.186967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.186982 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.187072 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:38.187288 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188330 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.189655 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190372 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190468 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190521 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.190612 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.190652 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
I20260810 20:59:38.190728 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46329 (local address 127.0.0.1:46548)
0810 20:59:38.186745 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.186783 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.186848 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.187076 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.187079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.187081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:38.187204 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.187207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188432 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.188432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.189574 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.189577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190473 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.190539 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190542 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.190598 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.190666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":88,"thread_start_us":49,"threads_started":1}
I20260810 20:59:38.190884 16311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:39595 (local address 127.0.0.1:44650)
0810 20:59:38.186899 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.186908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.186961 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.187266 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.187267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.187270 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:38.187321 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.187323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188335 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.188336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.189651 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.189653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190501 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.190555 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190560 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.190608 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.190833 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":241,"thread_start_us":163,"threads_started":1}
I20260810 20:59:38.190877 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.14.200.1:39595)
0810 20:59:38.186975 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.186979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.187019 (+ 40us) server_negotiation.cc:407] Connection header received
0810 20:59:38.187056 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.187058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.187073 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.187148 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:38.187399 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188165 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.189705 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190343 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190412 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190468 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.190650 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.190720 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":97,"threads_started":1}
I20260810 20:59:38.191757 15925 tablet_service.cc:1467] Processing CreateTablet for tablet b6c627c2c3ed4308a80f100b067892cf (DEFAULT_TABLE table=test-table [id=2594633267934bc7990321900e9a6618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:38.191852 16079 tablet_service.cc:1467] Processing CreateTablet for tablet b6c627c2c3ed4308a80f100b067892cf (DEFAULT_TABLE table=test-table [id=2594633267934bc7990321900e9a6618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:38.192111 15925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c627c2c3ed4308a80f100b067892cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:38.192214 16079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c627c2c3ed4308a80f100b067892cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:38.196120 16317 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Bootstrap starting.
I20260810 20:59:38.196271 16318 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: Bootstrap starting.
I20260810 20:59:38.196758 16310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.186303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:46239 (local address 127.0.0.1:32768)
0810 20:59:38.186745 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.186783 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.186847 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.187301 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.187302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.187303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:38.187330 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.187331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.188487 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.188488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.189392 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.189395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.190279 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.190280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.190371 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.190387 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.196499 (+ 6112us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.196584 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260810 20:59:38.197480 16317 tablet_bootstrap.cc:654] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:38.197861 16228 tablet_service.cc:1467] Processing CreateTablet for tablet b6c627c2c3ed4308a80f100b067892cf (DEFAULT_TABLE table=test-table [id=2594633267934bc7990321900e9a6618]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:38.198202 16228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c627c2c3ed4308a80f100b067892cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:38.198312 16318 tablet_bootstrap.cc:654] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:38.198697 16317 log.cc:824] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:38.199532 16317 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: No bootstrap required, opened a new log
I20260810 20:59:38.199576 16317 ts_tablet_manager.cc:1397] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 20:59:38.199565 16318 log.cc:824] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:38.200661 16318 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: No bootstrap required, opened a new log
I20260810 20:59:38.200728 16318 ts_tablet_manager.cc:1397] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 20:59:38.202483 16317 raft_consensus.cc:348] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.202761 16317 raft_consensus.cc:374] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:38.202841 16317 raft_consensus.cc:729] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6975dd5cd89f4a03a211e9500cf06445, State: Initialized, Role: FOLLOWER
I20260810 20:59:38.202983 16317 consensus_queue.cc:260] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.203450 16317 ts_tablet_manager.cc:1428] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 20:59:38.203454 16153 heartbeater.cc:503] Master 127.14.200.62:45529 was elected leader, sending a full tablet report...
I20260810 20:59:38.203723 16324 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: Bootstrap starting.
I20260810 20:59:38.205054 16318 raft_consensus.cc:348] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.205334 16324 tablet_bootstrap.cc:654] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:38.205327 16318 raft_consensus.cc:374] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:38.205377 16318 raft_consensus.cc:729] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf77524af7fb4bbda9d78febd1fa368e, State: Initialized, Role: FOLLOWER
I20260810 20:59:38.205484 16318 consensus_queue.cc:260] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.205941 16318 ts_tablet_manager.cc:1428] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 20:59:38.205948 16000 heartbeater.cc:503] Master 127.14.200.62:45529 was elected leader, sending a full tablet report...
I20260810 20:59:38.206513 16324 log.cc:824] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:38.207297 16324 tablet_bootstrap.cc:492] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: No bootstrap required, opened a new log
I20260810 20:59:38.207381 16324 ts_tablet_manager.cc:1397] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 20:59:38.210116 16324 raft_consensus.cc:348] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.210304 16324 raft_consensus.cc:374] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:38.210351 16324 raft_consensus.cc:729] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5179458724436bafacf676b63ac360, State: Initialized, Role: FOLLOWER
I20260810 20:59:38.210464 16324 consensus_queue.cc:260] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.210868 16324 ts_tablet_manager.cc:1428] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 20:59:38.210868 16302 heartbeater.cc:503] Master 127.14.200.62:45529 was elected leader, sending a full tablet report...
I20260810 20:59:38.238322 16323 raft_consensus.cc:482] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:38.238409 16323 raft_consensus.cc:504] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.239001 16323 leader_election.cc:288] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5179458724436bafacf676b63ac360 (127.14.200.3:46239), bf77524af7fb4bbda9d78febd1fa368e (127.14.200.1:39595)
I20260810 20:59:38.242654 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.239188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:39595 (local address 127.14.200.2:39431)
0810 20:59:38.239255 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.239265 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.239306 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.239460 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.239461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.239463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:38.239653 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.239656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.240487 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.240487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.241779 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.241782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.242425 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.242426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.242523 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.242528 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.242577 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.242608 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":58}
I20260810 20:59:38.242713 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.239247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39431 (local address 127.14.200.1:39595)
0810 20:59:38.239304 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.239307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.239317 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:38.239366 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.239367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.239377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.239430 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:38.239741 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.240409 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.241859 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.242350 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.242420 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.242428 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.242599 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.242636 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 20:59:38.243119 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.239286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60267 (local address 127.14.200.3:46239)
0810 20:59:38.239321 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.239323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.239475 (+ 152us) server_negotiation.cc:407] Connection header received
0810 20:59:38.239506 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.239507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.239517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.239564 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:38.239914 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.240538 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.242040 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.242646 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.242709 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.242716 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.243017 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.243061 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 20:59:38.243183 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.239218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:46239 (local address 127.14.200.2:60267)
0810 20:59:38.239394 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:38.239402 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:38.239463 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:38.239618 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:38.239619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:38.239620 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:38.239842 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.239844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.240599 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.240599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.241965 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:38.241967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.242801 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.242803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:38.242926 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.242939 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 20:59:38.243043 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 20:59:38.243079 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":124,"threads_started":1}
I20260810 20:59:38.243083 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c627c2c3ed4308a80f100b067892cf" candidate_uuid: "6975dd5cd89f4a03a211e9500cf06445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf77524af7fb4bbda9d78febd1fa368e" is_pre_election: true
I20260810 20:59:38.243314 15947 raft_consensus.cc:2454] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6975dd5cd89f4a03a211e9500cf06445 in term 0.
I20260810 20:59:38.243702 16027 leader_election.cc:302] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6975dd5cd89f4a03a211e9500cf06445, bf77524af7fb4bbda9d78febd1fa368e; no voters:
I20260810 20:59:38.243731 16250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c627c2c3ed4308a80f100b067892cf" candidate_uuid: "6975dd5cd89f4a03a211e9500cf06445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5179458724436bafacf676b63ac360" is_pre_election: true
I20260810 20:59:38.243854 16250 raft_consensus.cc:2454] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6975dd5cd89f4a03a211e9500cf06445 in term 0.
I20260810 20:59:38.243927 16323 raft_consensus.cc:2781] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:38.244001 16323 raft_consensus.cc:482] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:38.244038 16323 raft_consensus.cc:3037] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:38.245105 16323 raft_consensus.cc:504] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.245260 16323 leader_election.cc:288] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [CANDIDATE]: Term 1 election: Requested vote from peers ad5179458724436bafacf676b63ac360 (127.14.200.3:46239), bf77524af7fb4bbda9d78febd1fa368e (127.14.200.1:39595)
I20260810 20:59:38.245386 16250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c627c2c3ed4308a80f100b067892cf" candidate_uuid: "6975dd5cd89f4a03a211e9500cf06445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5179458724436bafacf676b63ac360"
I20260810 20:59:38.245461 16250 raft_consensus.cc:3037] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:38.245539 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c627c2c3ed4308a80f100b067892cf" candidate_uuid: "6975dd5cd89f4a03a211e9500cf06445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf77524af7fb4bbda9d78febd1fa368e"
I20260810 20:59:38.245621 15947 raft_consensus.cc:3037] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:38.246592 16250 raft_consensus.cc:2454] T b6c627c2c3ed4308a80f100b067892cf P ad5179458724436bafacf676b63ac360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6975dd5cd89f4a03a211e9500cf06445 in term 1.
I20260810 20:59:38.246716 15947 raft_consensus.cc:2454] T b6c627c2c3ed4308a80f100b067892cf P bf77524af7fb4bbda9d78febd1fa368e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6975dd5cd89f4a03a211e9500cf06445 in term 1.
I20260810 20:59:38.246824 16026 leader_election.cc:302] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6975dd5cd89f4a03a211e9500cf06445, ad5179458724436bafacf676b63ac360; no voters:
I20260810 20:59:38.246950 16323 raft_consensus.cc:2781] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:38.247121 16323 raft_consensus.cc:686] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [term 1 LEADER]: Becoming Leader. State: Replica: 6975dd5cd89f4a03a211e9500cf06445, State: Running, Role: LEADER
I20260810 20:59:38.247216 16323 consensus_queue.cc:237] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } }
I20260810 20:59:38.248013 15783 catalog_manager.cc:5495] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6975dd5cd89f4a03a211e9500cf06445 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "6975dd5cd89f4a03a211e9500cf06445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6975dd5cd89f4a03a211e9500cf06445" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5179458724436bafacf676b63ac360" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 46239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf77524af7fb4bbda9d78febd1fa368e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 39595 } health_report { overall_health: UNKNOWN } } }
W20260810 20:59:38.256942 16154 tablet.cc:2369] T b6c627c2c3ed4308a80f100b067892cf P 6975dd5cd89f4a03a211e9500cf06445: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:38.266672 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.259487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.14.200.1:39595)
0810 20:59:38.259544 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.259546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.259739 (+ 193us) server_negotiation.cc:407] Connection header received
0810 20:59:38.259772 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.259773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.259781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.259826 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.260420 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.260894 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.261580 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.261748 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.261794 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.264143 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.264150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.264151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.264157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.265780 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.266087 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.266089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.266089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.266142 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.266291 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.266291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.266292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.266394 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.266483 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.266545 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.266599 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:38.290686 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.284494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.14.200.1:39595)
0810 20:59:38.284535 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.284536 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.284603 (+ 67us) server_negotiation.cc:407] Connection header received
0810 20:59:38.284641 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.284642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.284649 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.284692 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.285190 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.285584 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.286183 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.286339 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.286381 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.288441 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.288442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.288442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.288449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.289606 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.289920 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.289921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.289921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.289982 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.290177 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.290178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.290178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.290270 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.290358 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.290525 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.290598 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[15734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:38.311569 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:38.305512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.14.200.1:39595)
0810 20:59:38.305548 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:38.305550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:38.305584 (+ 34us) server_negotiation.cc:407] Connection header received
0810 20:59:38.305647 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:38.305648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:38.305655 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:38.305701 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:38.306172 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.306622 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.307259 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:38.307417 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:38.307470 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:38.309507 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:38.309508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:38.309508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:38.309514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:38.310790 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.311060 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.311061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.311061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.311107 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:38.311244 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:38.311244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:38.311244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:38.311313 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:38.311372 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:38.311464 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 20:59:38.311513 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 20:59:38.312239 15898 tablet_service.cc:2908] Rejecting scan request for tablet b6c627c2c3ed4308a80f100b067892cf: Uninitialized: clean time has not yet been initialized
I20260810 20:59:38.313235 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15853
I20260810 20:59:38.320959 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16005
I20260810 20:59:38.328154 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16158
I20260810 20:59:38.334542 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 15747
2026-08-10T20:59:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1939 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[16345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[16345](info): set up 2 sockets
Aug 10 20:59:38 dist-test-slave-1x4g krb5kdc[16345](info): commencing operation
krb5kdc: starting...
W20260810 20:59:39.381760 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:39.495719 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:46653
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:46653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:39.591872 16359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:39.592056 16359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:39.592077 16359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:39.593184 16359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:39.593225 16359 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 20:59:39.593237 16359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:39.593250 16359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:39.593297 16359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:39.593313 16359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:39.594750 16359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:46653
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:46653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:39.594931 16359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:39.595162 16359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:39.599799 16367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.599799 16365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.599859 16364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:39.600020 16359 server_base.cc:1034] running on GCE node
I20260810 20:59:39.600231 16359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:39.600453 16359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:39.601555 16359 hybrid_clock.cc:648] HybridClock initialized: now 1786395579601538 us; error 33 us; skew 500 ppm
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:39.603981 16359 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:39.604362 16359 webserver.cc:460] Webserver started at http://127.14.200.62:38517/ using document root <none> and password file <none>
I20260810 20:59:39.604542 16359 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:39.604579 16359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:39.604653 16359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:39.605602 16359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "36bea595270c4e119de98f5205398626"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "ad77f2a5f1ff6d6435ced941ff6a06f8"
server_key_iv: "5adf24fb2abc21fc19724814b4b04902"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.605865 16359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "36bea595270c4e119de98f5205398626"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "ad77f2a5f1ff6d6435ced941ff6a06f8"
server_key_iv: "5adf24fb2abc21fc19724814b4b04902"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.607066 16359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:39.607661 16376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.607824 16359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:39.607896 16359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "36bea595270c4e119de98f5205398626"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "ad77f2a5f1ff6d6435ced941ff6a06f8"
server_key_iv: "5adf24fb2abc21fc19724814b4b04902"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.607949 16359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:39.622748 16359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:39.624387 16359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:39.624495 16359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:39.629118 16359 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:46653
I20260810 20:59:39.629202 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:46653 every 8 connection(s)
I20260810 20:59:39.629496 16359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:39.630095 16445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:39.631415 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16359
I20260810 20:59:39.631491 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:39.631647 15136 external_mini_cluster.cc:1428] Setting key 875dd88fdbd5474e1fe4f36bd5402cd2
I20260810 20:59:39.632663 16445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Bootstrap starting.
I20260810 20:59:39.633384 16445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:39.633638 16445 log.cc:824] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:39.634400 16445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: No bootstrap required, opened a new log
I20260810 20:59:39.636835 16445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } }
I20260810 20:59:39.637015 16445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:39.637042 16445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bea595270c4e119de98f5205398626, State: Initialized, Role: FOLLOWER
I20260810 20:59:39.637182 16445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } }
I20260810 20:59:39.637249 16445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:39.637271 16445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:39.637336 16445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:39.637844 16445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } }
I20260810 20:59:39.637969 16445 leader_election.cc:302] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36bea595270c4e119de98f5205398626; no voters:
I20260810 20:59:39.638131 16445 leader_election.cc:288] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:39.638180 16450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:39.638312 16450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [term 1 LEADER]: Becoming Leader. State: Replica: 36bea595270c4e119de98f5205398626, State: Running, Role: LEADER
I20260810 20:59:39.638362 16445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:39.638397 16450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } }
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:39.638684 16452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36bea595270c4e119de98f5205398626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } } }
I20260810 20:59:39.638767 16452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:39.638674 16450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bea595270c4e119de98f5205398626. Latest consensus state: current_term: 1 leader_uuid: "36bea595270c4e119de98f5205398626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bea595270c4e119de98f5205398626" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46653 } } }
I20260810 20:59:39.638863 16450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:39.639489 16457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:39.640461 16457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:39.641868 16457 catalog_manager.cc:1347] Generated new cluster ID: 241397cac6934eeea8103912641622e6
I20260810 20:59:39.641916 16457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:39.643208 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.632128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.14.200.62:46653)
0810 20:59:39.632318 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:39.632327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:39.632358 (+ 31us) server_negotiation.cc:407] Connection header received
0810 20:59:39.632896 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:39.632904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:39.632961 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:39.633139 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:39.633819 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.635098 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.635771 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.636324 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.636430 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.639125 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:39.639129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:39.639130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:39.639141 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:39.641521 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.641921 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.641923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.641924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.642021 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.642255 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.642256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.642256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.642538 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:39.642700 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:39.642866 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 20:59:39.643020 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260810 20:59:39.659868 16457 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:39.660517 16457 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:39.671239 16457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Generated new TSK 0
I20260810 20:59:39.671381 16457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:39.697386 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46653
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:39.776749 16475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:39.776953 16475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:39.776978 16475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:39.778133 16475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:39.778173 16475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:39.778192 16475 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:39.778247 16475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:39.779583 16475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:39.779824 16475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:39.780038 16475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:39.784752 16481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.784735 16484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.784905 16482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:39.785087 16475 server_base.cc:1034] running on GCE node
I20260810 20:59:39.785283 16475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:39.785482 16475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:39.786649 16475 hybrid_clock.cc:648] HybridClock initialized: now 1786395579786633 us; error 29 us; skew 500 ppm
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:39.789166 16475 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:39.789590 16475 webserver.cc:460] Webserver started at http://127.14.200.1:33011/ using document root <none> and password file <none>
I20260810 20:59:39.789817 16475 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:39.789891 16475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:39.790045 16475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:39.791081 16475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "b178e52819f1dd9d6ead05d381b437d1"
server_key_iv: "de927f885e993bf08be33a87d0949229"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.791487 16475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "b178e52819f1dd9d6ead05d381b437d1"
server_key_iv: "de927f885e993bf08be33a87d0949229"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.792834 16475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 20:59:39.793638 16491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.793812 16475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:39.793923 16475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "b178e52819f1dd9d6ead05d381b437d1"
server_key_iv: "de927f885e993bf08be33a87d0949229"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.794001 16475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:39.814772 16475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:39.816637 16475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:39.816876 16475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:39.817391 16475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:39.817461 16475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.817515 16475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:39.817572 16475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:39.825445 16475 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:45419
I20260810 20:59:39.825526 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:45419 every 8 connection(s)
I20260810 20:59:39.825820 16475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 20:59:39.828890 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.818008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37551 (local address 127.14.200.62:46653)
0810 20:59:39.818062 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:39.818064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:39.818272 (+ 208us) server_negotiation.cc:407] Connection header received
0810 20:59:39.818753 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:39.818756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:39.818769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:39.818826 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:39.819756 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.820546 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.822192 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.822396 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.822479 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.825449 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:39.825450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:39.825450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:39.825458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:39.827047 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.827597 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.827599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.827599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.827672 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.828081 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.828083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.828083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.828240 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:39.828304 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:39.828691 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 20:59:39.828801 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 20:59:39.829069 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.818008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.1:37551)
0810 20:59:39.818163 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:39.818196 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:39.818644 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:39.818958 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:39.818963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:39.819352 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:39.819636 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.819644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.820668 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.820669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.822093 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.822097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.822505 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.822506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.822676 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.823314 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:39.823324 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:39.825356 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:39.827154 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.827162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.827164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.827486 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.827763 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.827765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.827765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.827996 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.828333 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:39.828341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:39.828516 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 20:59:39.828654 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 20:59:39.828834 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260810 20:59:39.830740 16620 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46653
I20260810 20:59:39.830806 16620 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:39.830935 16620 heartbeater.cc:511] Master 127.14.200.62:46653 requested a full tablet report, sending...
I20260810 20:59:39.831521 16393 ts_manager.cc:194] Registered new tserver with Master: 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419)
I20260810 20:59:39.832237 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16475
I20260810 20:59:39.832345 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:39.832520 15136 external_mini_cluster.cc:1428] Setting key 9b52cf0233dbf7b744872ff9ab9e1dfb
I20260810 20:59:39.832487 16393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:37551
I20260810 20:59:39.838222 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.831108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37685 (local address 127.14.200.62:46653)
0810 20:59:39.831146 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:39.831148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:39.831362 (+ 214us) server_negotiation.cc:407] Connection header received
0810 20:59:39.831468 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:39.831470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:39.831481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:39.831514 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:39.831998 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.832661 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.833789 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.834102 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.834206 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.835187 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:39.835188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:39.835189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:39.835196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:39.836859 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.837341 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.837349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.837350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.837426 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.837659 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.837660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.837660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.837800 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:39.837886 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:39.838058 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 20:59:39.838131 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 20:59:39.838290 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.831146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.1:37685)
0810 20:59:39.831296 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:39.831308 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:39.831390 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:39.831584 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:39.831586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:39.831804 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:39.831883 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.831886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.833006 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.833006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.833676 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.833678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.834215 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.834216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.834317 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.834759 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:39.834764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:39.835069 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:39.836961 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.836962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.836963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.837236 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.837519 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.837520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.837520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.837586 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.837892 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:39.837893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:39.837939 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 20:59:39.838028 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 20:59:39.838132 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:39.866782 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46653
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:39.946170 16629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:39.946341 16629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:39.946362 16629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:39.947495 16629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:39.947536 16629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:39.947556 16629 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:39.947609 16629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:39.948983 16629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:39.949219 16629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:39.949432 16629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:39.954142 16637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.954145 16634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:39.954294 16635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:39.954552 16629 server_base.cc:1034] running on GCE node
I20260810 20:59:39.954741 16629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:39.954975 16629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:39.956131 16629 hybrid_clock.cc:648] HybridClock initialized: now 1786395579956118 us; error 25 us; skew 500 ppm
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:39.958575 16629 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:39.958983 16629 webserver.cc:460] Webserver started at http://127.14.200.2:41257/ using document root <none> and password file <none>
I20260810 20:59:39.959203 16629 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:39.959270 16629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:39.959421 16629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:39.960417 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "4265aff757804b30860d0b4bc4886da8"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "f91bf7d5d1f9ecb93d3364c06f830a5f"
server_key_iv: "8a41f57440cb2a2a4d7e0d3cb96f7136"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.960810 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "4265aff757804b30860d0b4bc4886da8"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "f91bf7d5d1f9ecb93d3364c06f830a5f"
server_key_iv: "8a41f57440cb2a2a4d7e0d3cb96f7136"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.962049 16629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:39.962711 16644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.962874 16629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:39.962975 16629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "4265aff757804b30860d0b4bc4886da8"
format_stamp: "Formatted at 2026-08-10 20:59:39 on dist-test-slave-1x4g"
server_key: "f91bf7d5d1f9ecb93d3364c06f830a5f"
server_key_iv: "8a41f57440cb2a2a4d7e0d3cb96f7136"
server_key_version: "encryptionkey@0"
I20260810 20:59:39.963080 16629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:39.975696 16629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:39.977918 16629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:39.978080 16629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:39.978545 16629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:39.978613 16629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.978653 16629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:39.978681 16629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:39.985852 16629 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:37693
I20260810 20:59:39.985940 16774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:37693 every 8 connection(s)
I20260810 20:59:39.986238 16629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 20:59:39 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:39.988958 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.979007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:51169 (local address 127.14.200.62:46653)
0810 20:59:39.979091 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:39.979093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:39.979228 (+ 135us) server_negotiation.cc:407] Connection header received
0810 20:59:39.979628 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:39.979629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:39.979643 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:39.979720 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:39.980497 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.981344 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.982944 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.983316 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.983397 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.986804 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:39.986806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:39.986806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:39.986813 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:39.987896 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.988223 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.988224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.988224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.988272 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.988469 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.988469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.988470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.988560 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:39.988608 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:39.988833 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 20:59:39.988902 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 20:59:39.989180 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.978966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.2:51169)
0810 20:59:39.979103 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:39.979137 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:39.979538 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:39.979765 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:39.979767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:39.980082 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:39.980366 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.980371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.981471 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.981472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.982850 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.982854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.983443 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.983444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.983608 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.984957 (+ 1349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:39.984969 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:39.986715 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:39.987963 (+ 1248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.987969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.987971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.988161 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.988326 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.988326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.988327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.988412 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.988611 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:39.988615 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:39.988710 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 20:59:39.988822 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 20:59:39.988954 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260810 20:59:39.989632 16775 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46653
I20260810 20:59:39.989730 16775 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:39.989884 16775 heartbeater.cc:511] Master 127.14.200.62:46653 requested a full tablet report, sending...
I20260810 20:59:39.990288 16394 ts_manager.cc:194] Registered new tserver with Master: 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693)
I20260810 20:59:39.990770 16394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:51169
I20260810 20:59:39.991287 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16629
I20260810 20:59:39.991372 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:39.991533 15136 external_mini_cluster.cc:1428] Setting key d331ddfffbd3c69317194eea45a92075
I20260810 20:59:39.996284 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.990664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54265 (local address 127.14.200.62:46653)
0810 20:59:39.990702 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:39.990704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:39.990774 (+ 70us) server_negotiation.cc:407] Connection header received
0810 20:59:39.990838 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:39.990839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:39.990847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:39.990894 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:39.991308 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.991982 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.992659 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.992989 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.993060 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.993958 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:39.993959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:39.993959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:39.993966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:39.995227 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.995582 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.995583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.995583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.995627 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:39.995827 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:39.995828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:39.995828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:39.995948 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:39.995995 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:39.996174 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 20:59:39.996231 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 20:59:39.996405 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:39.990607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.2:54265)
0810 20:59:39.990724 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:39.990734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:39.990787 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:39.990973 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:39.990974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:39.991166 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:39.991225 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.991227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.992075 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.992076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.992580 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:39.992581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:39.993061 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:39.993062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:39.993119 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:39.993547 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:39.993551 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:39.993861 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:39.995298 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.995299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.995299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.995516 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.995687 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:39.995688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:39.995688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:39.995770 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:39.996015 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:39.996016 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:39.996069 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 20:59:39.996159 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 20:59:39.996271 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:40.018965 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46653
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:40.099188 16784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:40.099373 16784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:40.099393 16784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:40.100445 16784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:40.100476 16784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:40.100494 16784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:40.100512 16784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:40.101783 16784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:46653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:40.101953 16784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:40.102113 16784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:40.106726 16794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:40.106757 16792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:40.106932 16791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:40.107095 16784 server_base.cc:1034] running on GCE node
I20260810 20:59:40.107306 16784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:40.107587 16784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:40.108736 16784 hybrid_clock.cc:648] HybridClock initialized: now 1786395580108719 us; error 33 us; skew 500 ppm
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395580, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:40.111701 16784 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:40.112093 16784 webserver.cc:460] Webserver started at http://127.14.200.3:37227/ using document root <none> and password file <none>
I20260810 20:59:40.112304 16784 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:40.112349 16784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:40.112484 16784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:40.113580 16784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "3b865726990244f9af50a5ab81762981"
format_stamp: "Formatted at 2026-08-10 20:59:40 on dist-test-slave-1x4g"
server_key: "8a00798b01a84124ee9b1579d0fe9dcd"
server_key_iv: "6e5d153e0b9bbad92437cbf9e9f8b25e"
server_key_version: "encryptionkey@0"
I20260810 20:59:40.113945 16784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "3b865726990244f9af50a5ab81762981"
format_stamp: "Formatted at 2026-08-10 20:59:40 on dist-test-slave-1x4g"
server_key: "8a00798b01a84124ee9b1579d0fe9dcd"
server_key_iv: "6e5d153e0b9bbad92437cbf9e9f8b25e"
server_key_version: "encryptionkey@0"
I20260810 20:59:40.115213 16784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:40.115876 16802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.116058 16784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:40.116145 16784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "3b865726990244f9af50a5ab81762981"
format_stamp: "Formatted at 2026-08-10 20:59:40 on dist-test-slave-1x4g"
server_key: "8a00798b01a84124ee9b1579d0fe9dcd"
server_key_iv: "6e5d153e0b9bbad92437cbf9e9f8b25e"
server_key_version: "encryptionkey@0"
I20260810 20:59:40.116216 16784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:40.131783 16784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:40.133983 16784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:40.134155 16784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:40.134684 16784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:40.134750 16784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.134783 16784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:40.134812 16784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395580, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:40.142287 16784 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:42261
I20260810 20:59:40.142620 16784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:40.142696 16930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:42261 every 8 connection(s)
I20260810 20:59:40.143903 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 16784
I20260810 20:59:40.144011 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:40.144181 15136 external_mini_cluster.cc:1428] Setting key a02a53a12b826b0ec4b13f53fad4b7e7
I20260810 20:59:40.145097 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.135146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:44421 (local address 127.14.200.62:46653)
0810 20:59:40.135221 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.135224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.135470 (+ 246us) server_negotiation.cc:407] Connection header received
0810 20:59:40.136163 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.136165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.136176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.136235 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.137171 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.138001 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.139312 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.139583 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.139683 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.141864 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.141866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.141866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.141873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.143530 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.143908 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.143909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.143910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.143978 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.144302 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.144303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.144303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.144428 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.144522 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.144918 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.145004 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 20:59:40.145285 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.135116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.3:44421)
0810 20:59:40.135320 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.135367 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.136049 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.136311 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.136315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.136736 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:40.137061 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.137068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.138120 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.138121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.139238 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.139241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.139708 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.139710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.139831 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.140325 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:40.140334 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:40.141767 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:40.143603 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.143608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.143609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.143835 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.144068 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.144070 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.144070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.144209 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.144563 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:40.144570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:40.144733 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.144881 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.145069 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260810 20:59:40.145756 16931 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46653
I20260810 20:59:40.145834 16931 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:40.146021 16931 heartbeater.cc:511] Master 127.14.200.62:46653 requested a full tablet report, sending...
I20260810 20:59:40.146400 16393 ts_manager.cc:194] Registered new tserver with Master: 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261)
I20260810 20:59:40.146919 16393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:44421
I20260810 20:59:40.151788 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.146815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:33223 (local address 127.14.200.62:46653)
0810 20:59:40.146872 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.146873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.146946 (+ 73us) server_negotiation.cc:407] Connection header received
0810 20:59:40.147025 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.147026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.147036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.147104 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.147452 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.147992 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.148636 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.148865 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.148923 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.149662 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.149663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.149663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.149669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.151020 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.151290 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.151291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.151291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.151335 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.151475 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.151476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.151476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.151560 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.151602 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.151684 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.151736 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 20:59:40.151796 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.146762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.14.200.3:33223)
0810 20:59:40.146886 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.146897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.146959 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.147112 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.147113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.147317 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:40.147379 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.147381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.148080 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.148080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.148568 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.148569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.148929 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.148930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.149004 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.149318 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:40.149321 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:40.149593 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:40.151084 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.151084 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.151085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.151229 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.151388 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.151389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.151389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.151439 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.151592 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:40.151592 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:40.151624 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.151667 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.151735 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260810 20:59:40.156015 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:40.178557 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.172207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.14.200.62:46653)
0810 20:59:40.172258 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.172260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.172274 (+ 14us) server_negotiation.cc:407] Connection header received
0810 20:59:40.172294 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.172295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.172303 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.172352 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.172949 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.173434 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.174105 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.174290 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.174337 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.176285 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.176286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.176287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.176292 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.177708 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.178000 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.178001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.178001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.178050 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.178201 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.178201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.178202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.178290 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.178360 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.178437 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.178490 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 20:59:40.180037 16393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:40.181707 16393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:40.187909 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.183900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:45419 (local address 127.0.0.1:34600)
0810 20:59:40.184064 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.184103 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.184170 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.184568 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.184569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.184572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.184629 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.184631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.186052 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.186053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.186926 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.186928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187619 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.187696 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.187710 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.187779 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.187835 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":57,"threads_started":1}
I20260810 20:59:40.188009 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.184180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.14.200.2:37693)
0810 20:59:40.184346 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.184350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.184359 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:40.184402 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.184404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.184416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.184511 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.184813 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.185516 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.186898 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187672 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187747 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.187803 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.187866 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.187907 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260810 20:59:40.188180 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.184036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.14.200.1:45419)
0810 20:59:40.184365 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.184370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.184382 (+ 12us) server_negotiation.cc:407] Connection header received
0810 20:59:40.184425 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.184427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.184444 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.184707 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.184816 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.185931 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187007 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187534 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187618 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.187681 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.188040 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.188090 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":78,"threads_started":1}
I20260810 20:59:40.188241 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.183923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37693 (local address 127.0.0.1:49404)
0810 20:59:40.184210 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.184234 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.184288 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.184525 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.184528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.184531 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.184710 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.184712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.185604 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.185605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.186831 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.186833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187744 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.187802 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.187805 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.187853 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.187885 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":100,"threads_started":1}
I20260810 20:59:40.188514 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.183929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:42261 (local address 127.0.0.1:38786)
0810 20:59:40.184364 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.184373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.184409 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.184727 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.184728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.184729 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.184764 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.184769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.186008 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.186009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.187028 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.187030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187932 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.188162 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.188167 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.188408 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.188448 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":98,"threads_started":1}
I20260810 20:59:40.188520 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.184158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.14.200.3:42261)
0810 20:59:40.184531 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.184534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.184543 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:40.184577 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.184579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.184591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.184661 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.184891 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.185915 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187108 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.187787 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.187849 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.187897 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.188370 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.188423 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":89,"threads_started":1}
I20260810 20:59:40.188865 16545 tablet_service.cc:1467] Processing CreateTablet for tablet 8f57fb861fc542e08a20c8e663a07553 (DEFAULT_TABLE table=test-table [id=e290eb09700c4417a06f7ffae6b36053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.189134 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 8f57fb861fc542e08a20c8e663a07553 (DEFAULT_TABLE table=test-table [id=e290eb09700c4417a06f7ffae6b36053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.189276 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f57fb861fc542e08a20c8e663a07553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.189276 16858 tablet_service.cc:1467] Processing CreateTablet for tablet 8f57fb861fc542e08a20c8e663a07553 (DEFAULT_TABLE table=test-table [id=e290eb09700c4417a06f7ffae6b36053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.189491 16699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f57fb861fc542e08a20c8e663a07553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.189571 16858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f57fb861fc542e08a20c8e663a07553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.192175 16957 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Bootstrap starting.
I20260810 20:59:40.193121 16958 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Bootstrap starting.
I20260810 20:59:40.193255 16957 tablet_bootstrap.cc:654] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.193642 16957 log.cc:824] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:40.193825 16958 tablet_bootstrap.cc:654] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.194145 16958 log.cc:824] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:40.194547 16957 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: No bootstrap required, opened a new log
I20260810 20:59:40.194629 16957 ts_tablet_manager.cc:1397] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 20:59:40.194893 16961 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Bootstrap starting.
I20260810 20:59:40.194981 16958 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: No bootstrap required, opened a new log
I20260810 20:59:40.195057 16958 ts_tablet_manager.cc:1397] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:40.195639 16961 tablet_bootstrap.cc:654] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.196043 16961 log.cc:824] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:40.196884 16961 tablet_bootstrap.cc:492] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: No bootstrap required, opened a new log
I20260810 20:59:40.196955 16961 ts_tablet_manager.cc:1397] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:40.197088 16958 raft_consensus.cc:348] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.197242 16958 raft_consensus.cc:374] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.197281 16958 raft_consensus.cc:729] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4265aff757804b30860d0b4bc4886da8, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.197414 16958 consensus_queue.cc:260] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.197781 16775 heartbeater.cc:503] Master 127.14.200.62:46653 was elected leader, sending a full tablet report...
I20260810 20:59:40.197701 16957 raft_consensus.cc:348] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.197844 16957 raft_consensus.cc:374] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.197908 16957 raft_consensus.cc:729] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b865726990244f9af50a5ab81762981, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.198016 16957 consensus_queue.cc:260] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.198102 16958 ts_tablet_manager.cc:1428] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 20:59:40.198273 16957 ts_tablet_manager.cc:1428] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260810 20:59:40.198302 16931 heartbeater.cc:503] Master 127.14.200.62:46653 was elected leader, sending a full tablet report...
I20260810 20:59:40.199527 16961 raft_consensus.cc:348] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.199638 16961 raft_consensus.cc:374] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.199657 16961 raft_consensus.cc:729] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53cade8edb7f4400a38bdb4ee9dcfe4c, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.199772 16961 consensus_queue.cc:260] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.200168 16961 ts_tablet_manager.cc:1428] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 20:59:40.200258 16620 heartbeater.cc:503] Master 127.14.200.62:46653 was elected leader, sending a full tablet report...
I20260810 20:59:40.202327 16964 raft_consensus.cc:482] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:40.202400 16964 raft_consensus.cc:504] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.202646 16964 leader_election.cc:288] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693), 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419)
I20260810 20:59:40.205459 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.202803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:45419 (local address 127.14.200.3:41857)
0810 20:59:40.202840 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.202851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.202907 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.203189 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.203189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.203191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.203250 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.203252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.203865 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.203866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.204584 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.204586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.205195 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.205196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.205287 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.205291 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.205339 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.205378 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 20:59:40.205653 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.203023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:41857 (local address 127.14.200.1:45419)
0810 20:59:40.203088 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.203090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.203096 (+ 6us) server_negotiation.cc:407] Connection header received
0810 20:59:40.203114 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.203114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.203121 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.203184 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.203321 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.203779 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.204669 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.205115 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.205215 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.205225 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.205549 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.205595 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 20:59:40.205967 16567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f57fb861fc542e08a20c8e663a07553" candidate_uuid: "3b865726990244f9af50a5ab81762981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" is_pre_election: true
I20260810 20:59:40.206099 16567 raft_consensus.cc:2454] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b865726990244f9af50a5ab81762981 in term 0.
I20260810 20:59:40.206377 16806 leader_election.cc:302] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 53cade8edb7f4400a38bdb4ee9dcfe4c; no voters:
I20260810 20:59:40.206423 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.202925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:39083 (local address 127.14.200.2:37693)
0810 20:59:40.202973 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.202975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.203203 (+ 228us) server_negotiation.cc:407] Connection header received
0810 20:59:40.203265 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.203266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.203275 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.203345 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.203611 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.204050 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.205289 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.205876 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.205964 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.205973 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.206322 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.206360 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 20:59:40.206498 16966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.202803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37693 (local address 127.14.200.3:39083)
0810 20:59:40.203121 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.203131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.203215 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.203348 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.203349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.203350 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.203528 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.203530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.204130 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.204131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.205165 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.205168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.206051 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.206053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.206244 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.206249 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.206302 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.206335 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260810 20:59:40.206586 16964 raft_consensus.cc:2781] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:40.206661 16964 raft_consensus.cc:482] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:40.206697 16964 raft_consensus.cc:3037] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.206856 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f57fb861fc542e08a20c8e663a07553" candidate_uuid: "3b865726990244f9af50a5ab81762981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4265aff757804b30860d0b4bc4886da8" is_pre_election: true
I20260810 20:59:40.207000 16722 raft_consensus.cc:2454] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b865726990244f9af50a5ab81762981 in term 0.
I20260810 20:59:40.208086 16964 raft_consensus.cc:504] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.208223 16964 leader_election.cc:288] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [CANDIDATE]: Term 1 election: Requested vote from peers 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693), 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419)
I20260810 20:59:40.208384 16567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f57fb861fc542e08a20c8e663a07553" candidate_uuid: "3b865726990244f9af50a5ab81762981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c"
I20260810 20:59:40.208384 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f57fb861fc542e08a20c8e663a07553" candidate_uuid: "3b865726990244f9af50a5ab81762981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4265aff757804b30860d0b4bc4886da8"
I20260810 20:59:40.208458 16722 raft_consensus.cc:3037] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.208458 16567 raft_consensus.cc:3037] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.209489 16567 raft_consensus.cc:2454] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b865726990244f9af50a5ab81762981 in term 1.
I20260810 20:59:40.209685 16806 leader_election.cc:302] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 53cade8edb7f4400a38bdb4ee9dcfe4c; no voters:
I20260810 20:59:40.209829 16964 raft_consensus.cc:2781] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:40.209892 16722 raft_consensus.cc:2454] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b865726990244f9af50a5ab81762981 in term 1.
I20260810 20:59:40.210001 16964 raft_consensus.cc:686] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 1 LEADER]: Becoming Leader. State: Replica: 3b865726990244f9af50a5ab81762981, State: Running, Role: LEADER
I20260810 20:59:40.210090 16964 consensus_queue.cc:237] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.210791 16393 catalog_manager.cc:5495] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b865726990244f9af50a5ab81762981 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "3b865726990244f9af50a5ab81762981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } health_report { overall_health: HEALTHY } } }
I20260810 20:59:40.217136 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.214837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.14.200.3:42261)
0810 20:59:40.214882 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.214883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.214910 (+ 27us) server_negotiation.cc:407] Connection header received
0810 20:59:40.214981 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.214981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.214994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.215048 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:40.215225 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.215632 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.216361 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.216527 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.216573 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.216882 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.216971 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.217035 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.217077 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.217094 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 20:59:40.219429 16567 raft_consensus.cc:1261] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Refusing update from remote peer 3b865726990244f9af50a5ab81762981: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.219537 16722 raft_consensus.cc:1261] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Refusing update from remote peer 3b865726990244f9af50a5ab81762981: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.219823 16964 consensus_queue.cc:1035] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.220081 16967 consensus_queue.cc:1035] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.223201 16970 mvcc.cc:205] Tried to move back new op lower bound from 7317076296576045056 to 7317076296541175808. Current Snapshot: MvccSnapshot[applied={T|T < 7317076296576045056}]
I20260810 20:59:40.223896 16972 mvcc.cc:205] Tried to move back new op lower bound from 7317076296576045056 to 7317076296541175808. Current Snapshot: MvccSnapshot[applied={T|T < 7317076296576045056 or (T in {7317076296576045056})}]
I20260810 20:59:40.224488 16971 mvcc.cc:205] Tried to move back new op lower bound from 7317076296576045056 to 7317076296541175808. Current Snapshot: MvccSnapshot[applied={T|T < 7317076296576045056 or (T in {7317076296576045056})}]
I20260810 20:59:40.228488 16394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:40.228592 16394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:40.229475 16394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Sending DeleteTablet for 3 replicas of tablet 8f57fb861fc542e08a20c8e663a07553
I20260810 20:59:40.229920 16858 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f57fb861fc542e08a20c8e663a07553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38786
I20260810 20:59:40.230432 16394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:40.230544 16394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:40.230854 16982 tablet_replica.cc:330] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: stopping tablet replica
I20260810 20:59:40.230922 16982 raft_consensus.cc:2229] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:40.231014 16982 raft_consensus.cc:2258] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.232020 16545 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f57fb861fc542e08a20c8e663a07553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:34600
I20260810 20:59:40.232389 16983 tablet_replica.cc:330] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: stopping tablet replica
I20260810 20:59:40.232496 16983 raft_consensus.cc:2229] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:40.232560 16699 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f57fb861fc542e08a20c8e663a07553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49404
I20260810 20:59:40.233039 16983 raft_consensus.cc:2258] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.233566 16983 ts_tablet_manager.cc:1905] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.233574 16982 ts_tablet_manager.cc:1905] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.234375 16545 tablet_service.cc:1467] Processing CreateTablet for tablet 2613dc5e8a42406a93c237528c3b0314 (DEFAULT_TABLE table=test-table [id=8b86620121104a34977f1542ff658a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.234412 16858 tablet_service.cc:1467] Processing CreateTablet for tablet 2613dc5e8a42406a93c237528c3b0314 (DEFAULT_TABLE table=test-table [id=8b86620121104a34977f1542ff658a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.234512 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2613dc5e8a42406a93c237528c3b0314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.234549 16858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2613dc5e8a42406a93c237528c3b0314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.236054 16982 ts_tablet_manager.cc:1918] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:40.236119 16982 log.cc:1196] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/8f57fb861fc542e08a20c8e663a07553
I20260810 20:59:40.236346 16957 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Bootstrap starting.
I20260810 20:59:40.236397 16982 ts_tablet_manager.cc:1939] T 8f57fb861fc542e08a20c8e663a07553 P 3b865726990244f9af50a5ab81762981: Deleting consensus metadata
I20260810 20:59:40.237052 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 2613dc5e8a42406a93c237528c3b0314 (DEFAULT_TABLE table=test-table [id=8b86620121104a34977f1542ff658a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:40.237232 16699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2613dc5e8a42406a93c237528c3b0314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.237340 16957 tablet_bootstrap.cc:654] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.237581 16984 tablet_replica.cc:330] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: stopping tablet replica
I20260810 20:59:40.237792 16984 raft_consensus.cc:2229] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:40.237978 16984 raft_consensus.cc:2258] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.238075 16379 catalog_manager.cc:4841] TS 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261): tablet 8f57fb861fc542e08a20c8e663a07553 (table test-table [id=e290eb09700c4417a06f7ffae6b36053]) successfully deleted
I20260810 20:59:40.238713 16984 ts_tablet_manager.cc:1905] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.239100 16958 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Bootstrap starting.
I20260810 20:59:40.239298 16961 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Bootstrap starting.
I20260810 20:59:40.239753 16983 ts_tablet_manager.cc:1918] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:40.239871 16983 log.cc:1196] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/8f57fb861fc542e08a20c8e663a07553
I20260810 20:59:40.240145 16958 tablet_bootstrap.cc:654] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.240178 16983 ts_tablet_manager.cc:1939] T 8f57fb861fc542e08a20c8e663a07553 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting consensus metadata
I20260810 20:59:40.240502 16961 tablet_bootstrap.cc:654] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.240656 16378 catalog_manager.cc:4841] TS 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419): tablet 8f57fb861fc542e08a20c8e663a07553 (table test-table [id=e290eb09700c4417a06f7ffae6b36053]) successfully deleted
I20260810 20:59:40.241420 16957 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: No bootstrap required, opened a new log
I20260810 20:59:40.241479 16957 ts_tablet_manager.cc:1397] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 20:59:40.241617 16957 raft_consensus.cc:348] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.241695 16957 raft_consensus.cc:374] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.241719 16957 raft_consensus.cc:729] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b865726990244f9af50a5ab81762981, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.241770 16957 consensus_queue.cc:260] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.241873 16957 ts_tablet_manager.cc:1428] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:40.242030 16984 ts_tablet_manager.cc:1918] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:40.242097 16984 log.cc:1196] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/8f57fb861fc542e08a20c8e663a07553
I20260810 20:59:40.242375 16984 ts_tablet_manager.cc:1939] T 8f57fb861fc542e08a20c8e663a07553 P 4265aff757804b30860d0b4bc4886da8: Deleting consensus metadata
I20260810 20:59:40.242795 16377 catalog_manager.cc:4841] TS 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693): tablet 8f57fb861fc542e08a20c8e663a07553 (table test-table [id=e290eb09700c4417a06f7ffae6b36053]) successfully deleted
I20260810 20:59:40.243014 16958 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: No bootstrap required, opened a new log
I20260810 20:59:40.243045 16958 ts_tablet_manager.cc:1397] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 20:59:40.243168 16958 raft_consensus.cc:348] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.243208 16958 raft_consensus.cc:374] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.243220 16958 raft_consensus.cc:729] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4265aff757804b30860d0b4bc4886da8, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.243253 16958 consensus_queue.cc:260] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.243338 16958 ts_tablet_manager.cc:1428] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.245157 16961 tablet_bootstrap.cc:492] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: No bootstrap required, opened a new log
I20260810 20:59:40.245214 16961 ts_tablet_manager.cc:1397] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260810 20:59:40.245357 16961 raft_consensus.cc:348] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.245424 16961 raft_consensus.cc:374] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.245445 16961 raft_consensus.cc:729] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53cade8edb7f4400a38bdb4ee9dcfe4c, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.245491 16961 consensus_queue.cc:260] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.245581 16961 ts_tablet_manager.cc:1428] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.257791 16965 raft_consensus.cc:482] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:40.257876 16965 raft_consensus.cc:504] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.258217 16965 leader_election.cc:288] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693), 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261)
I20260810 20:59:40.261900 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.258537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:42221 (local address 127.14.200.3:42261)
0810 20:59:40.258582 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.258584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.258691 (+ 107us) server_negotiation.cc:407] Connection header received
0810 20:59:40.258720 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.258721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.258731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.258778 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.258928 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.259415 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.261018 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.261571 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.261644 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.261654 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.261797 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.261835 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 20:59:40.261938 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.258366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:42261 (local address 127.14.200.1:42221)
0810 20:59:40.258620 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.258629 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.258681 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.258779 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.258780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.258781 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.258847 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.258848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.259505 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.259506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.260919 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.260921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.261673 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.261674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.261723 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.261726 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.261788 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.261828 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":63,"thread_start_us":63,"threads_started":1}
I20260810 20:59:40.262264 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2613dc5e8a42406a93c237528c3b0314" candidate_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b865726990244f9af50a5ab81762981" is_pre_election: true
I20260810 20:59:40.262400 16878 raft_consensus.cc:2454] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53cade8edb7f4400a38bdb4ee9dcfe4c in term 0.
I20260810 20:59:40.262565 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.258341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37693 (local address 127.14.200.1:33173)
0810 20:59:40.258405 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.258415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.258451 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.258587 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.258588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.258590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.258656 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.258658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.259613 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.259614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.260917 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.260921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.262323 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.262324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.262403 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.262408 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.262460 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.262502 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":2}
I20260810 20:59:40.262719 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.258421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33173 (local address 127.14.200.2:37693)
0810 20:59:40.258462 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.258464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.258474 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:40.258499 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.258500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.258510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.258563 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.258755 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.259530 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.261691 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.262230 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.262298 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.262305 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.262619 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.262657 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 20:59:40.262828 16494 leader_election.cc:302] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 53cade8edb7f4400a38bdb4ee9dcfe4c; no voters:
I20260810 20:59:40.262951 16965 raft_consensus.cc:2781] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:40.262980 16965 raft_consensus.cc:482] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:40.263000 16965 raft_consensus.cc:3037] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.262997 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2613dc5e8a42406a93c237528c3b0314" candidate_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4265aff757804b30860d0b4bc4886da8" is_pre_election: true
I20260810 20:59:40.263087 16722 raft_consensus.cc:2454] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53cade8edb7f4400a38bdb4ee9dcfe4c in term 0.
I20260810 20:59:40.263731 16965 raft_consensus.cc:504] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.263865 16965 leader_election.cc:288] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [CANDIDATE]: Term 1 election: Requested vote from peers 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693), 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261)
I20260810 20:59:40.264004 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2613dc5e8a42406a93c237528c3b0314" candidate_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4265aff757804b30860d0b4bc4886da8"
I20260810 20:59:40.264016 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2613dc5e8a42406a93c237528c3b0314" candidate_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b865726990244f9af50a5ab81762981"
I20260810 20:59:40.264081 16722 raft_consensus.cc:3037] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.264099 16878 raft_consensus.cc:3037] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.264694 16878 raft_consensus.cc:2454] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53cade8edb7f4400a38bdb4ee9dcfe4c in term 1.
I20260810 20:59:40.264871 16494 leader_election.cc:302] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 53cade8edb7f4400a38bdb4ee9dcfe4c; no voters:
I20260810 20:59:40.264959 16965 raft_consensus.cc:2781] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:40.265092 16722 raft_consensus.cc:2454] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53cade8edb7f4400a38bdb4ee9dcfe4c in term 1.
I20260810 20:59:40.265108 16965 raft_consensus.cc:686] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 LEADER]: Becoming Leader. State: Replica: 53cade8edb7f4400a38bdb4ee9dcfe4c, State: Running, Role: LEADER
I20260810 20:59:40.265182 16965 consensus_queue.cc:237] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.265825 16394 catalog_manager.cc:5495] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1). New cstate: current_term: 1 leader_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } health_report { overall_health: UNKNOWN } } }
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395579, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:40.292761 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.285353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46653 (local address 127.0.0.1:48344)
0810 20:59:40.285492 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.285503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.285561 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.285683 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.285684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.285966 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:40.286006 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.286007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.286896 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.286897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.287453 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.287454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.287941 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.287941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.287980 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.288561 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:40.288565 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:40.289973 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:40.291799 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.291806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.291808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.292010 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.292205 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:40.292205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:40.292206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:40.292260 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:40.292506 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:40.292512 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:40.292550 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.292599 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.292675 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260810 20:59:40.293066 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.285462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.14.200.62:46653)
0810 20:59:40.285501 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.285504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.285569 (+ 65us) server_negotiation.cc:407] Connection header received
0810 20:59:40.285611 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.285612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.285621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.285674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.286084 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.286799 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.287532 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.287841 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.287919 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.290058 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.290060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.290060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.290067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.291696 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.292079 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.292080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.292080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.292140 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.292320 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.292320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.292321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.292445 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.292512 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.292640 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.292731 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":22}
I20260810 20:59:40.295047 16394 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:48344:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 20:59:40.295280 16394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:40.296633 16858 tablet_service.cc:1467] Processing CreateTablet for tablet 788393841b7b43ae9980f6cca4cc713d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2041199e8839420f92463b374122686e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:40.296860 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 788393841b7b43ae9980f6cca4cc713d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2041199e8839420f92463b374122686e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:40.296887 16545 tablet_service.cc:1467] Processing CreateTablet for tablet 788393841b7b43ae9980f6cca4cc713d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2041199e8839420f92463b374122686e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:40.297022 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 788393841b7b43ae9980f6cca4cc713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.297024 16699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 788393841b7b43ae9980f6cca4cc713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.298743 16958 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: Bootstrap starting.
I20260810 20:59:40.298741 16961 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: Bootstrap starting.
I20260810 20:59:40.298784 16858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 788393841b7b43ae9980f6cca4cc713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:40.299531 16961 tablet_bootstrap.cc:654] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.299569 16958 tablet_bootstrap.cc:654] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.300616 16961 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: No bootstrap required, opened a new log
I20260810 20:59:40.300659 16961 ts_tablet_manager.cc:1397] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 20:59:40.300872 16957 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: Bootstrap starting.
I20260810 20:59:40.300926 16958 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: No bootstrap required, opened a new log
I20260810 20:59:40.300959 16958 ts_tablet_manager.cc:1397] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:40.300977 16961 raft_consensus.cc:348] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.301043 16961 raft_consensus.cc:374] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.301060 16961 raft_consensus.cc:729] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53cade8edb7f4400a38bdb4ee9dcfe4c, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.301097 16961 consensus_queue.cc:260] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.301121 16958 raft_consensus.cc:348] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.301170 16961 ts_tablet_manager.cc:1428] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.301194 16958 raft_consensus.cc:374] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.301219 16958 raft_consensus.cc:729] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4265aff757804b30860d0b4bc4886da8, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.301229 16957 tablet_bootstrap.cc:654] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:40.301260 16958 consensus_queue.cc:260] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.301275 16965 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.301329 16965 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.301349 16958 ts_tablet_manager.cc:1428] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:40.301518 16963 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.301608 16963 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.302232 16957 tablet_bootstrap.cc:492] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: No bootstrap required, opened a new log
I20260810 20:59:40.302296 16957 ts_tablet_manager.cc:1397] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:40.302469 16957 raft_consensus.cc:348] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.302541 16957 raft_consensus.cc:374] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:40.302564 16957 raft_consensus.cc:729] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b865726990244f9af50a5ab81762981, State: Initialized, Role: FOLLOWER
I20260810 20:59:40.302608 16957 consensus_queue.cc:260] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.302688 16957 ts_tablet_manager.cc:1428] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:40.302798 16979 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.302989 16979 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 20:59:40.326653 16621 tablet.cc:2369] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 20:59:40.393178 16932 tablet.cc:2369] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 20:59:40.487123 16776 tablet.cc:2369] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 20:59:40.490079 16963 raft_consensus.cc:482] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:40.490182 16963 raft_consensus.cc:504] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.490556 16963 leader_election.cc:288] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419), 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261)
I20260810 20:59:40.494921 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.490888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:58317 (local address 127.14.200.1:45419)
0810 20:59:40.490936 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.490938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.490952 (+ 14us) server_negotiation.cc:407] Connection header received
0810 20:59:40.491044 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.491045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.491055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.491117 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.491343 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.492100 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.493947 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.494516 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.494589 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.494596 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.494814 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.494852 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 20:59:40.494921 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.490835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:56213 (local address 127.14.200.3:42261)
0810 20:59:40.490896 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.490899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.490965 (+ 66us) server_negotiation.cc:407] Connection header received
0810 20:59:40.491045 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.491046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.491055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.491117 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:40.491429 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.492105 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.493995 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.494515 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.494588 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.494596 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.494814 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.494851 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 20:59:40.495082 17000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.490754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:42261 (local address 127.14.200.2:56213)
0810 20:59:40.490887 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.490900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.490969 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.491140 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.491141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.491145 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.491351 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.491353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.492186 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.492187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.493919 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.493922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.494599 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.494600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.494732 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.494739 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.494797 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.494847 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":51,"thread_start_us":59,"threads_started":1}
I20260810 20:59:40.495280 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.490767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:45419 (local address 127.14.200.2:58317)
0810 20:59:40.490894 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.490903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.490969 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.491140 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.491141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.491143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:40.491252 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.491255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.492186 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.492187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.493845 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.493849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.494597 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.494599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.494732 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.494740 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.494797 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.494847 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":109,"thread_start_us":45,"threads_started":1}
I20260810 20:59:40.495385 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "788393841b7b43ae9980f6cca4cc713d" candidate_uuid: "4265aff757804b30860d0b4bc4886da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b865726990244f9af50a5ab81762981" is_pre_election: true
I20260810 20:59:40.495481 16878 raft_consensus.cc:2454] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4265aff757804b30860d0b4bc4886da8 in term 0.
I20260810 20:59:40.495535 16567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "788393841b7b43ae9980f6cca4cc713d" candidate_uuid: "4265aff757804b30860d0b4bc4886da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" is_pre_election: true
I20260810 20:59:40.495620 16567 raft_consensus.cc:2454] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4265aff757804b30860d0b4bc4886da8 in term 0.
I20260810 20:59:40.496042 16647 leader_election.cc:302] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 4265aff757804b30860d0b4bc4886da8; no voters:
I20260810 20:59:40.496199 16963 raft_consensus.cc:2781] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:40.496246 16963 raft_consensus.cc:482] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:40.496261 16963 raft_consensus.cc:3037] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.497359 16963 raft_consensus.cc:504] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.497514 16963 leader_election.cc:288] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [CANDIDATE]: Term 1 election: Requested vote from peers 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419), 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261)
I20260810 20:59:40.497721 16567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "788393841b7b43ae9980f6cca4cc713d" candidate_uuid: "4265aff757804b30860d0b4bc4886da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c"
I20260810 20:59:40.497720 16878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "788393841b7b43ae9980f6cca4cc713d" candidate_uuid: "4265aff757804b30860d0b4bc4886da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b865726990244f9af50a5ab81762981"
I20260810 20:59:40.497799 16567 raft_consensus.cc:3037] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.497799 16878 raft_consensus.cc:3037] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:40.498762 16878 raft_consensus.cc:2454] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4265aff757804b30860d0b4bc4886da8 in term 1.
I20260810 20:59:40.498762 16567 raft_consensus.cc:2454] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4265aff757804b30860d0b4bc4886da8 in term 1.
I20260810 20:59:40.498931 16647 leader_election.cc:302] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b865726990244f9af50a5ab81762981, 4265aff757804b30860d0b4bc4886da8; no voters:
I20260810 20:59:40.499059 16963 raft_consensus.cc:2781] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:40.499214 16963 raft_consensus.cc:686] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [term 1 LEADER]: Becoming Leader. State: Replica: 4265aff757804b30860d0b4bc4886da8, State: Running, Role: LEADER
I20260810 20:59:40.499282 16963 consensus_queue.cc:237] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } }
I20260810 20:59:40.499622 17002 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: TxnStatusTablet state changed. Reason: New leader 4265aff757804b30860d0b4bc4886da8. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.499701 17002 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:40.500005 17004 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 20:59:40.500012 16394 catalog_manager.cc:5495] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4265aff757804b30860d0b4bc4886da8 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } health_report { overall_health: UNKNOWN } } }
I20260810 20:59:40.500140 17004 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 20:59:40.531387 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.528344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37693 (local address 127.0.0.1:49406)
0810 20:59:40.528424 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.528438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.528502 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.528718 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.528719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.528720 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:40.528887 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.528890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.529831 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.529832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.530415 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.530417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.530891 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.530892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.530954 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.530971 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.531242 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.531250 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.531289 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.531330 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 20:59:40.531392 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.528430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.14.200.2:37693)
0810 20:59:40.528504 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.528506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.528517 (+ 11us) server_negotiation.cc:407] Connection header received
0810 20:59:40.528579 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.528580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.528603 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.528676 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:40.528985 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.529746 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.530492 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.530788 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.530871 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.531057 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.531178 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.531249 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.531297 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.531317 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 20:59:40.532634 16567 raft_consensus.cc:1261] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Refusing update from remote peer 4265aff757804b30860d0b4bc4886da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.532706 16878 raft_consensus.cc:1261] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Refusing update from remote peer 4265aff757804b30860d0b4bc4886da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.532975 17002 consensus_queue.cc:1035] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.533146 16963 consensus_queue.cc:1035] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.533802 16965 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: TxnStatusTablet state changed. Reason: New leader 4265aff757804b30860d0b4bc4886da8. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.533891 16965 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.533839 16979 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: TxnStatusTablet state changed. Reason: New leader 4265aff757804b30860d0b4bc4886da8. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.533936 16979 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.534415 16963 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.534509 16963 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:40.535068 17002 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.535154 17002 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 4265aff757804b30860d0b4bc4886da8: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:40.535414 16965 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.535491 16965 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 53cade8edb7f4400a38bdb4ee9dcfe4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.535557 16979 tablet_replica.cc:439] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4265aff757804b30860d0b4bc4886da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53cade8edb7f4400a38bdb4ee9dcfe4c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 45419 } } peers { permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 } } peers { permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 } } }
I20260810 20:59:40.535652 16979 tablet_replica.cc:442] T 788393841b7b43ae9980f6cca4cc713d P 3b865726990244f9af50a5ab81762981: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:40.539662 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.536690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.14.200.1:45419)
0810 20:59:40.536725 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.536727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.536809 (+ 82us) server_negotiation.cc:407] Connection header received
0810 20:59:40.536909 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.536910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.536930 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.537024 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:40.537143 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.537952 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.538755 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.539056 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.539140 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.539274 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.539388 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.539478 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.539560 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.539581 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 20:59:40.544618 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.541972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:43971 (local address 127.14.200.2:37693)
0810 20:59:40.542035 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.542037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.542143 (+ 106us) server_negotiation.cc:407] Connection header received
0810 20:59:40.542199 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.542199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.542206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.542261 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:40.542502 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.542905 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.544028 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.544192 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.544234 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.544433 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.544455 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.544490 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.544561 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.544576 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 20:59:40.544750 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.541916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37693 (local address 127.14.200.1:43971)
0810 20:59:40.542080 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.542090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.542147 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.542273 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.542273 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.542332 (+ 59us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:40.542444 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.542446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.542980 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.542980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.543941 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.543944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.544275 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.544276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.544343 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.544359 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.544510 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.544511 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.544554 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.544596 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260810 20:59:40.547070 16878 raft_consensus.cc:1261] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Refusing update from remote peer 53cade8edb7f4400a38bdb4ee9dcfe4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.547076 16722 raft_consensus.cc:1261] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Refusing update from remote peer 53cade8edb7f4400a38bdb4ee9dcfe4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:40.547333 16965 consensus_queue.cc:1035] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4265aff757804b30860d0b4bc4886da8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.547487 16965 consensus_queue.cc:1035] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b865726990244f9af50a5ab81762981" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:40.558205 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.554646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:37903 (local address 127.14.200.1:45419)
0810 20:59:40.554710 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.554712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.554980 (+ 268us) server_negotiation.cc:407] Connection header received
0810 20:59:40.555044 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.555045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.555054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.555106 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:40.555436 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.555930 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.557139 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.557444 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.557520 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.557839 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.557915 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.557976 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.558103 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.558129 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 20:59:40.558461 17017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.554561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:45419 (local address 127.14.200.2:37903)
0810 20:59:40.554911 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:40.554920 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:40.554993 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:40.555110 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:40.555111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:40.555156 (+ 45us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:40.555327 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.555329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.556047 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.556048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.557060 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:40.557064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.557555 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.557556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:40.557713 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.557734 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:40.558000 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:40.558002 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 20:59:40.558322 (+ 320us) client_negotiation.cc:240] Negotiation successful
0810 20:59:40.558369 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":220,"threads_started":1}
I20260810 20:59:40.572369 16394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:40.572487 16394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:40.573127 16394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36bea595270c4e119de98f5205398626: Sending DeleteTablet for 3 replicas of tablet 2613dc5e8a42406a93c237528c3b0314
I20260810 20:59:40.573477 16545 tablet_service.cc:1514] Processing DeleteTablet for tablet 2613dc5e8a42406a93c237528c3b0314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:34600
I20260810 20:59:40.573573 16983 tablet_replica.cc:330] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: stopping tablet replica
I20260810 20:59:40.573616 16983 raft_consensus.cc:2229] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:40.573693 16983 raft_consensus.cc:2258] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.574776 16699 tablet_service.cc:1514] Processing DeleteTablet for tablet 2613dc5e8a42406a93c237528c3b0314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49404
I20260810 20:59:40.574887 16984 tablet_replica.cc:330] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: stopping tablet replica
I20260810 20:59:40.574929 16984 raft_consensus.cc:2229] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:40.574958 16984 raft_consensus.cc:2258] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.575363 16984 ts_tablet_manager.cc:1905] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.575492 16858 tablet_service.cc:1514] Processing DeleteTablet for tablet 2613dc5e8a42406a93c237528c3b0314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:40 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38786
I20260810 20:59:40.575575 16982 tablet_replica.cc:330] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: stopping tablet replica
I20260810 20:59:40.575618 16982 raft_consensus.cc:2229] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:40.575646 16982 raft_consensus.cc:2258] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:40.575901 16983 ts_tablet_manager.cc:1905] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.577255 16984 ts_tablet_manager.cc:1918] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:40.577307 16984 log.cc:1196] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/2613dc5e8a42406a93c237528c3b0314
I20260810 20:59:40.577574 16984 ts_tablet_manager.cc:1939] T 2613dc5e8a42406a93c237528c3b0314 P 4265aff757804b30860d0b4bc4886da8: Deleting consensus metadata
I20260810 20:59:40.577976 16377 catalog_manager.cc:4841] TS 4265aff757804b30860d0b4bc4886da8 (127.14.200.2:37693): tablet 2613dc5e8a42406a93c237528c3b0314 (table test-table [id=8b86620121104a34977f1542ff658a43]) successfully deleted
I20260810 20:59:40.578066 16983 ts_tablet_manager.cc:1918] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:40.578114 16983 log.cc:1196] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/2613dc5e8a42406a93c237528c3b0314
I20260810 20:59:40.578308 16983 ts_tablet_manager.cc:1939] T 2613dc5e8a42406a93c237528c3b0314 P 53cade8edb7f4400a38bdb4ee9dcfe4c: Deleting consensus metadata
I20260810 20:59:40.579530 16378 catalog_manager.cc:4841] TS 53cade8edb7f4400a38bdb4ee9dcfe4c (127.14.200.1:45419): tablet 2613dc5e8a42406a93c237528c3b0314 (table test-table [id=8b86620121104a34977f1542ff658a43]) successfully deleted
I20260810 20:59:40.579795 16982 ts_tablet_manager.cc:1905] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:40.581328 16982 ts_tablet_manager.cc:1918] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:40.581414 16982 log.cc:1196] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/2613dc5e8a42406a93c237528c3b0314
I20260810 20:59:40.581629 16982 ts_tablet_manager.cc:1939] T 2613dc5e8a42406a93c237528c3b0314 P 3b865726990244f9af50a5ab81762981: Deleting consensus metadata
I20260810 20:59:40.581962 16379 catalog_manager.cc:4841] TS 3b865726990244f9af50a5ab81762981 (127.14.200.3:42261): tablet 2613dc5e8a42406a93c237528c3b0314 (table test-table [id=8b86620121104a34977f1542ff658a43]) successfully deleted
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[16345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:40.592128 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.579450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.14.200.1:45419)
0810 20:59:40.579485 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.579487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.582243 (+ 2756us) server_negotiation.cc:407] Connection header received
0810 20:59:40.582297 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.582298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.582312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.582365 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.583108 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.583873 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.584934 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.585209 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.585294 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.588708 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.588715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.588715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.588723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.590719 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.591157 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.591160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.591161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.591231 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.591513 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.591514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.591514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.591675 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.591802 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.591934 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.592011 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 20:59:40.592490 16611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34610
I20260810 20:59:40.601132 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:40.593679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.14.200.62:46653)
0810 20:59:40.593727 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:40.593729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:40.593739 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:40.593785 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:40.593786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:40.593797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:40.593876 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:40.594501 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.595303 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.596406 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:40.596737 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:40.596921 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:40.598236 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:40.598238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:40.598238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:40.598245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:40.599825 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.600259 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.600261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.600261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.600325 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:40.600561 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:40.600562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:40.600563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:40.600697 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:40.600923 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:40.600968 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 20:59:40.601046 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 20:59:40.601537 16394 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48352
I20260810 20:59:40.602276 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16475
I20260810 20:59:40.617449 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16629
I20260810 20:59:40.625531 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16784
I20260810 20:59:40.632988 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 16359
2026-08-10T20:59:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2298 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[17031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[17031](info): set up 2 sockets
Aug 10 20:59:40 dist-test-slave-1x4g krb5kdc[17031](info): commencing operation
krb5kdc: starting...
W20260810 20:59:41.702034 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:41 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:41.815550 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45667
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:41.891530 17044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:41.891733 17044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:41.891765 17044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:41.892787 17044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:41.892863 17044 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 20:59:41.892884 17044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:41.892903 17044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:41.892942 17044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:41.892966 17044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:41.894474 17044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45667
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:41.894712 17044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:41.894910 17044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:41.899614 17052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:41.899647 17050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:41.899650 17049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:41.899716 17044 server_base.cc:1034] running on GCE node
I20260810 20:59:41.899974 17044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:41.900231 17044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:41.901386 17044 hybrid_clock.cc:648] HybridClock initialized: now 1786395581901375 us; error 37 us; skew 500 ppm
Aug 10 20:59:41 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395581, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:41.903688 17044 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:41.904109 17044 webserver.cc:460] Webserver started at http://127.14.200.62:37535/ using document root <none> and password file <none>
I20260810 20:59:41.904278 17044 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:41.904328 17044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:41.904422 17044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:41.905395 17044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "4e1d74c86f15433e9287cd823dd57ef1"
format_stamp: "Formatted at 2026-08-10 20:59:41 on dist-test-slave-1x4g"
server_key: "0a973bb779ba7091910a61b2bbd96924"
server_key_iv: "52620464e64b93caecb1db26c5ba93d0"
server_key_version: "encryptionkey@0"
I20260810 20:59:41.905714 17044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "4e1d74c86f15433e9287cd823dd57ef1"
format_stamp: "Formatted at 2026-08-10 20:59:41 on dist-test-slave-1x4g"
server_key: "0a973bb779ba7091910a61b2bbd96924"
server_key_iv: "52620464e64b93caecb1db26c5ba93d0"
server_key_version: "encryptionkey@0"
I20260810 20:59:41.906845 17044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:41.907445 17059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:41.907617 17044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:41.907697 17044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "4e1d74c86f15433e9287cd823dd57ef1"
format_stamp: "Formatted at 2026-08-10 20:59:41 on dist-test-slave-1x4g"
server_key: "0a973bb779ba7091910a61b2bbd96924"
server_key_iv: "52620464e64b93caecb1db26c5ba93d0"
server_key_version: "encryptionkey@0"
I20260810 20:59:41.907747 17044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:41.918080 17044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:41.920222 17044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:41.920321 17044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:41.924747 17044 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45667
I20260810 20:59:41.924921 17131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45667 every 8 connection(s)
I20260810 20:59:41.925251 17044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:41.925871 17132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:41.927981 17132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Bootstrap starting.
I20260810 20:59:41.928530 17132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:41.928845 17132 log.cc:824] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:41.929490 17132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: No bootstrap required, opened a new log
I20260810 20:59:41.930142 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17044
I20260810 20:59:41.930236 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:41.930409 15136 external_mini_cluster.cc:1428] Setting key 20bd119d53905abbbb204b9891f3430e
I20260810 20:59:41.932163 17132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } }
I20260810 20:59:41.932309 17132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:41.932344 17132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1d74c86f15433e9287cd823dd57ef1, State: Initialized, Role: FOLLOWER
I20260810 20:59:41.932467 17132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } }
I20260810 20:59:41.932549 17132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:41.932582 17132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:41.932628 17132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:41.933274 17132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } }
I20260810 20:59:41.933360 17132 leader_election.cc:302] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e1d74c86f15433e9287cd823dd57ef1; no voters:
I20260810 20:59:41.933557 17132 leader_election.cc:288] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:41.933594 17137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:41.933702 17137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [term 1 LEADER]: Becoming Leader. State: Replica: 4e1d74c86f15433e9287cd823dd57ef1, State: Running, Role: LEADER
I20260810 20:59:41.933802 17132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:41.933780 17137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } }
I20260810 20:59:41.934238 17139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e1d74c86f15433e9287cd823dd57ef1. Latest consensus state: current_term: 1 leader_uuid: "4e1d74c86f15433e9287cd823dd57ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } } }
I20260810 20:59:41.934331 17139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:41.934518 17138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e1d74c86f15433e9287cd823dd57ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e1d74c86f15433e9287cd823dd57ef1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45667 } } }
I20260810 20:59:41.934624 17138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:41.934732 17145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:41.935241 17145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:41.937615 17145 catalog_manager.cc:1347] Generated new cluster ID: 9f530995a5ff48a88371d4d2e0719d57
I20260810 20:59:41.937667 17145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 20:59:41 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:41.941995 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:41.930911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.14.200.62:45667)
0810 20:59:41.931100 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:41.931107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:41.931128 (+ 21us) server_negotiation.cc:407] Connection header received
0810 20:59:41.932146 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:41.932152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:41.932192 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:41.932291 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:41.933068 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:41.934318 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:41.935319 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:41.935917 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:41.936040 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:41.938517 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:41.938521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:41.938522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:41.938533 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:41.940412 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:41.940799 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:41.940802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:41.940802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:41.940938 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:41.941153 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:41.941154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:41.941154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:41.941475 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:41.941621 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:41.941688 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 20:59:41.941818 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260810 20:59:41.947753 17145 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:41.948446 17145 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:41.951798 17145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Generated new TSK 0
I20260810 20:59:41.951951 17145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:41.979184 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45667
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:42.058799 17158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:42.059005 17158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:42.059053 17158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:42.060142 17158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:42.060199 17158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:42.060233 17158 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 20:59:42.060276 17158 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:42.060312 17158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:42.061734 17158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:42.061980 17158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:42.062196 17158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:42.067183 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:42.067276 17158 server_base.cc:1034] running on GCE node
W20260810 20:59:42.067160 17166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:42.067167 17164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:42.067576 17158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:42.067764 17158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:42.068845 17158 hybrid_clock.cc:648] HybridClock initialized: now 1786395582068816 us; error 36 us; skew 500 ppm
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:42.071182 17158 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:42.071587 17158 webserver.cc:460] Webserver started at http://127.14.200.1:38377/ using document root <none> and password file <none>
I20260810 20:59:42.071815 17158 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:42.071857 17158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:42.071992 17158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:42.072990 17158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "34bf1494d086757741f7ae4ad7df4db8"
server_key_iv: "c1eefc23ff14d2d01eaf9f6c93956688"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.073341 17158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "34bf1494d086757741f7ae4ad7df4db8"
server_key_iv: "c1eefc23ff14d2d01eaf9f6c93956688"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.074664 17158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:42.075380 17173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.075541 17158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:42.075663 17158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "34bf1494d086757741f7ae4ad7df4db8"
server_key_iv: "c1eefc23ff14d2d01eaf9f6c93956688"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.075736 17158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:42.085579 17158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:42.087247 17158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:42.087402 17158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:42.087893 17158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:42.087956 17158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.087997 17158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:42.088021 17158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:42.095223 17158 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:43517
I20260810 20:59:42.095315 17297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:43517 every 8 connection(s)
I20260810 20:59:42.095577 17158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 20:59:42.098716 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.088374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37763 (local address 127.14.200.62:45667)
0810 20:59:42.088450 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.088452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.088700 (+ 248us) server_negotiation.cc:407] Connection header received
0810 20:59:42.089393 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.089407 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.089421 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.089496 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.090339 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.091133 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.092702 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.093030 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.093099 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.095709 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.095710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.095710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.095718 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.097356 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.097766 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.097768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.097768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.097829 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.098079 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.098080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.098081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.098189 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.098255 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.098528 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.098622 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 20:59:42.098865 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.088300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.1:37763)
0810 20:59:42.088553 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.088604 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.089276 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.089575 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.089579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.089952 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.090238 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.090244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.091236 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.091237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.092611 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.092615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.093132 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.093133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.093340 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.093801 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.093811 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.095611 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.097441 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.097448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.097450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.097673 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.097897 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.097898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.097898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.098014 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.098259 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.098265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.098387 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.098503 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.098666 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260810 20:59:42.099357 17298 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45667
I20260810 20:59:42.099443 17298 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:42.099609 17298 heartbeater.cc:511] Master 127.14.200.62:45667 requested a full tablet report, sending...
I20260810 20:59:42.100101 17077 ts_manager.cc:194] Registered new tserver with Master: 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517)
I20260810 20:59:42.101006 17077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:37763
I20260810 20:59:42.104161 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17158
I20260810 20:59:42.104247 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:42.104496 15136 external_mini_cluster.cc:1428] Setting key 1e953ebefaac5f5d6bdd8460fdf56792
I20260810 20:59:42.105573 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.100387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:56237 (local address 127.14.200.62:45667)
0810 20:59:42.100452 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.100454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.100466 (+ 12us) server_negotiation.cc:407] Connection header received
0810 20:59:42.100547 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.100547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.100554 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.100593 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.100946 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.101563 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.102188 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.102378 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.102433 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.103158 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.103159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.103159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.103165 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.104348 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.104808 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.104810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.104810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.104874 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.105079 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.105080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.105080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.105186 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.105259 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.105423 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.105492 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 20:59:42.105581 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.100269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.1:56237)
0810 20:59:42.100401 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.100412 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.100480 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.100606 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.100606 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.100807 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.100877 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.100878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.101667 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.101668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.102117 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.102119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.102447 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.102448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.102510 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.102827 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.102830 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.103085 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.104481 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.104482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.104483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.104707 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.104950 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.104951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.104951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.105016 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.105252 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.105253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.105297 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.105395 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.105493 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:42.129608 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45667
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:42.208297 17308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:42.208491 17308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:42.208523 17308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:42.209556 17308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:42.209589 17308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:42.209609 17308 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 20:59:42.209619 17308 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:42.209635 17308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:42.211054 17308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:42.211241 17308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:42.211454 17308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:42.216133 17316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:42.216140 17318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:42.216172 17315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:42.216298 17308 server_base.cc:1034] running on GCE node
I20260810 20:59:42.216459 17308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:42.216807 17308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:42.217965 17308 hybrid_clock.cc:648] HybridClock initialized: now 1786395582217950 us; error 32 us; skew 500 ppm
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:42.220098 17308 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:42.220476 17308 webserver.cc:460] Webserver started at http://127.14.200.2:34505/ using document root <none> and password file <none>
I20260810 20:59:42.220685 17308 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:42.220746 17308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:42.220908 17308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:42.221841 17308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "0cde8918cf1046409d0605c06a6a3b96"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "b310738198895f3642cc8aebebcda807"
server_key_iv: "a4e0cd995763bc0384600e55d01c1811"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.222245 17308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "0cde8918cf1046409d0605c06a6a3b96"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "b310738198895f3642cc8aebebcda807"
server_key_iv: "a4e0cd995763bc0384600e55d01c1811"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.223527 17308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:42.224215 17325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.224381 17308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:42.224493 17308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "0cde8918cf1046409d0605c06a6a3b96"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "b310738198895f3642cc8aebebcda807"
server_key_iv: "a4e0cd995763bc0384600e55d01c1811"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.224568 17308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:42.234191 17308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:42.235891 17308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:42.236032 17308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:42.236521 17308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:42.236574 17308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.236624 17308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:42.236675 17308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:42.243943 17308 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:43311
I20260810 20:59:42.244024 17452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:43311 every 8 connection(s)
I20260810 20:59:42.244338 17308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 20:59:42.246400 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.236912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54575 (local address 127.14.200.62:45667)
0810 20:59:42.236971 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.236974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.237233 (+ 259us) server_negotiation.cc:407] Connection header received
0810 20:59:42.237908 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.237909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.237920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.237977 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.238852 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.239339 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.240879 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.241064 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.241136 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.243839 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.243841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.243841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.243849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.245387 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.245732 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.245733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.245733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.245783 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.245944 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.245945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.245945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.246034 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.246079 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.246283 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.246340 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 20:59:42.246608 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.236918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.2:54575)
0810 20:59:42.237083 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.237130 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.237784 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.238069 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.238073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.238452 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.238740 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.238747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.239447 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.239448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.240814 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.240817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.241162 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.241163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.241279 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.241852 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.241861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.243671 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.245457 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.245462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.245463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.245668 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.245821 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.245821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.245822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.245898 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.246085 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.246090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.246189 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.246268 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.246374 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260810 20:59:42.247087 17453 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45667
I20260810 20:59:42.247180 17453 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:42.247344 17453 heartbeater.cc:511] Master 127.14.200.62:45667 requested a full tablet report, sending...
I20260810 20:59:42.247745 17079 ts_manager.cc:194] Registered new tserver with Master: 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:42.248122 17079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:54575
I20260810 20:59:42.253129 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.248079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60049 (local address 127.14.200.62:45667)
0810 20:59:42.248112 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.248114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.248235 (+ 121us) server_negotiation.cc:407] Connection header received
0810 20:59:42.248293 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.248293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.248304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.248370 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.248662 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.249146 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.249823 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.250024 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.250093 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.250829 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.250830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.250830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.250837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.251916 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.252301 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.252302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.252302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.252370 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.252566 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.252567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.252567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.252706 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.252779 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.252971 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.253048 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 20:59:42.253149 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.248033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.2:60049)
0810 20:59:42.248157 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.248165 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.248243 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.248381 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.248382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.248532 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.248585 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.248586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.249230 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.249231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.249742 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.249743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.250093 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.250094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.250166 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.250490 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.250494 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.250736 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.252017 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.252018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.252018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.252183 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.252444 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.252445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.252446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.252505 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.252789 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.252790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.252834 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.252939 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.253050 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260810 20:59:42.254060 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17308
I20260810 20:59:42.254171 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:42.254367 15136 external_mini_cluster.cc:1428] Setting key 993a59abb2a3751c68e6a0c1c1e7822d
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:42.278878 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45667
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:42.357334 17462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:42.357517 17462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:42.357539 17462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:42.358603 17462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:42.358639 17462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:42.358657 17462 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 20:59:42.358701 17462 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 20:59:42.358727 17462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:42.360083 17462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:45667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:42.360268 17462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:42.360461 17462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:42.365059 17469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:42.365070 17472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:42.365069 17470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:42.365439 17462 server_base.cc:1034] running on GCE node
I20260810 20:59:42.365600 17462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:42.365769 17462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:42.366904 17462 hybrid_clock.cc:648] HybridClock initialized: now 1786395582366895 us; error 30 us; skew 500 ppm
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:42.369268 17462 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:42.369671 17462 webserver.cc:460] Webserver started at http://127.14.200.3:39179/ using document root <none> and password file <none>
I20260810 20:59:42.369875 17462 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:42.369936 17462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:42.370062 17462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:42.371012 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "4b9d3753b36e5b429784550ead5a7edc"
server_key_iv: "8105ca5151b646784a9ed7980d783067"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.371362 17462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "4b9d3753b36e5b429784550ead5a7edc"
server_key_iv: "8105ca5151b646784a9ed7980d783067"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.372567 17462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 20:59:42.373374 17479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.373553 17462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.373651 17462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3"
format_stamp: "Formatted at 2026-08-10 20:59:42 on dist-test-slave-1x4g"
server_key: "4b9d3753b36e5b429784550ead5a7edc"
server_key_iv: "8105ca5151b646784a9ed7980d783067"
server_key_version: "encryptionkey@0"
I20260810 20:59:42.373736 17462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:42.397599 17462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:42.399816 17462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:42.399966 17462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:42.400429 17462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:42.400494 17462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.400533 17462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:42.400553 17462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:42.407384 17462 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:39171
I20260810 20:59:42.407516 17606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:39171 every 8 connection(s)
I20260810 20:59:42.407766 17462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:42.410233 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.400960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:57519 (local address 127.14.200.62:45667)
0810 20:59:42.401003 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.401005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.401215 (+ 210us) server_negotiation.cc:407] Connection header received
0810 20:59:42.401678 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.401679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.401693 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.401774 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.402434 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.403227 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.404807 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.405166 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.405260 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.407784 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.407786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.407786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.407792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.409046 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.409398 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.409399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.409399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.409446 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.409639 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.409640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.409640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.409731 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.409776 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.410058 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.410142 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 20:59:42.410261 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.400946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.3:57519)
0810 20:59:42.401104 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.401137 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.401577 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.401820 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.401823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.402039 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.402325 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.402332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.403332 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.403333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.404691 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.404694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.405270 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.405271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.405419 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.405936 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.405943 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.407700 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.409131 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.409137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.409139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.409334 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.409503 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.409504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.409504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.409592 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.409793 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.409799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.409918 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.410000 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.410107 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260810 20:59:42.410686 17607 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45667
I20260810 20:59:42.410763 17607 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:42.410964 17607 heartbeater.cc:511] Master 127.14.200.62:45667 requested a full tablet report, sending...
I20260810 20:59:42.411389 17077 ts_manager.cc:194] Registered new tserver with Master: ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171)
I20260810 20:59:42.411806 17077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:57519
I20260810 20:59:42.413393 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17462
I20260810 20:59:42.413504 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:42.413699 15136 external_mini_cluster.cc:1428] Setting key 61b71d7999447168bdae7f24877054f6
I20260810 20:59:42.414848 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 20:59:42.417629 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.411968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:33417 (local address 127.14.200.62:45667)
0810 20:59:42.412007 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.412009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.412063 (+ 54us) server_negotiation.cc:407] Connection header received
0810 20:59:42.412129 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.412130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.412138 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.412181 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.412503 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.413068 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.414030 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.414331 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.414404 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.415213 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.415215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.415215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.415221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.416507 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.416975 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.416976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.416976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.417030 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.417232 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.417232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.417232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.417313 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.417356 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.417522 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.417576 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 20:59:42.417733 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.411835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.14.200.3:33417)
0810 20:59:42.412001 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.412012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.412080 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.412186 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.412187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.412364 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.412432 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.412434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.413293 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.413294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.413937 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.413940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.414411 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.414412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.414486 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.414844 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.414848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.415135 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.416629 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.416630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.416630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.416911 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.417099 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.417100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.417101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.417172 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.417367 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.417368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.417418 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.417509 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.417613 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:42.437630 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.431090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.14.200.62:45667)
0810 20:59:42.431136 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.431138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.431148 (+ 10us) server_negotiation.cc:407] Connection header received
0810 20:59:42.431218 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.431219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.431228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.431291 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.431796 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.432206 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.432926 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.433137 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.433255 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.435414 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.435415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.435416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.435422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.436495 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.436874 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.436879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.436880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.436938 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.437135 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.437136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.437136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.437237 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.437339 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.437448 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.437518 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 20:59:42.438989 17077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:42.440382 17077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:42.446966 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.14.200.2:43311)
0810 20:59:42.443036 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.443039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.443047 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:42.443079 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.443080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.443092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.443150 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.443329 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444427 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.445868 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.446548 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.446613 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.446674 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.446846 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.446880 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":108,"threads_started":1}
I20260810 20:59:42.446971 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43311 (local address 127.0.0.1:35556)
0810 20:59:42.442782 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.442820 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.442866 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.443163 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.443167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.443169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.443247 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.443248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444501 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.444502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.445794 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.445797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.446618 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.446619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.446732 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.446742 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.446823 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.446886 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":55,"thread_start_us":97,"threads_started":1}
I20260810 20:59:42.446970 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39171 (local address 127.0.0.1:51258)
0810 20:59:42.442786 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.442824 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.442879 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.443622 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.443623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.443625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.443665 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.443666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444743 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.444744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.445693 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.445696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.446614 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.446615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.446727 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.446742 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.446822 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.446886 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":36,"thread_start_us":62,"threads_started":1}
I20260810 20:59:42.447259 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.14.200.3:39171)
0810 20:59:42.443425 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.443429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.443448 (+ 19us) server_negotiation.cc:407] Connection header received
0810 20:59:42.443480 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.443482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.443496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.443565 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.443734 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444664 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.445784 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.446502 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.446578 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.446656 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.447118 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.447152 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":75,"threads_started":1}
I20260810 20:59:42.447726 17380 tablet_service.cc:1467] Processing CreateTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e (DEFAULT_TABLE table=test-table [id=f731fcfe0d7340ab92821b3e575b56ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.447891 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.0.0.1:57768)
0810 20:59:42.443269 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.443278 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.443323 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.443521 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.443522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.443523 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.443569 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.443571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444384 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.444385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.445793 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.445797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.447665 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.447666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.447744 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.447747 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.447796 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.447831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":64,"thread_start_us":38,"threads_started":1}
I20260810 20:59:42.448109 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388b5b7995564dc0b8dbd1f3c7289e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.448086 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e (DEFAULT_TABLE table=test-table [id=f731fcfe0d7340ab92821b3e575b56ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.448241 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.442826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.14.200.1:43517)
0810 20:59:42.443006 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.443026 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.443362 (+ 336us) server_negotiation.cc:407] Connection header received
0810 20:59:42.443409 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.443411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.443426 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.443516 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.443624 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.444297 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.446712 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.447520 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.447579 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.447635 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.448091 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.448154 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260810 20:59:42.448416 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388b5b7995564dc0b8dbd1f3c7289e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.448829 17229 tablet_service.cc:1467] Processing CreateTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e (DEFAULT_TABLE table=test-table [id=f731fcfe0d7340ab92821b3e575b56ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.449163 17229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388b5b7995564dc0b8dbd1f3c7289e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.451185 17628 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Bootstrap starting.
I20260810 20:59:42.451893 17628 tablet_bootstrap.cc:654] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.452235 17628 log.cc:824] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:42.452550 17631 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Bootstrap starting.
I20260810 20:59:42.452565 17629 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Bootstrap starting.
I20260810 20:59:42.453018 17628 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: No bootstrap required, opened a new log
I20260810 20:59:42.453070 17628 ts_tablet_manager.cc:1397] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:42.453200 17631 tablet_bootstrap.cc:654] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.453547 17631 log.cc:824] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:42.454219 17631 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: No bootstrap required, opened a new log
I20260810 20:59:42.454300 17631 ts_tablet_manager.cc:1397] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:42.454780 17628 raft_consensus.cc:348] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.454900 17628 raft_consensus.cc:374] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.454921 17628 raft_consensus.cc:729] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.455056 17628 consensus_queue.cc:260] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.455338 17628 ts_tablet_manager.cc:1428] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 20:59:42.455379 17453 heartbeater.cc:503] Master 127.14.200.62:45667 was elected leader, sending a full tablet report...
I20260810 20:59:42.456169 17631 raft_consensus.cc:348] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.456310 17631 raft_consensus.cc:374] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.456338 17631 raft_consensus.cc:729] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.456382 17629 tablet_bootstrap.cc:654] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.456442 17631 consensus_queue.cc:260] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.456713 17629 log.cc:824] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:42.456751 17298 heartbeater.cc:503] Master 127.14.200.62:45667 was elected leader, sending a full tablet report...
I20260810 20:59:42.456715 17631 ts_tablet_manager.cc:1428] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 20:59:42.457634 17629 tablet_bootstrap.cc:492] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: No bootstrap required, opened a new log
I20260810 20:59:42.457708 17629 ts_tablet_manager.cc:1397] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 20:59:42.458951 17629 raft_consensus.cc:348] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.459081 17629 raft_consensus.cc:374] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.459123 17629 raft_consensus.cc:729] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.459221 17629 consensus_queue.cc:260] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.459453 17629 ts_tablet_manager.cc:1428] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:42.459535 17607 heartbeater.cc:503] Master 127.14.200.62:45667 was elected leader, sending a full tablet report...
I20260810 20:59:42.469523 17636 raft_consensus.cc:482] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:42.469607 17636 raft_consensus.cc:504] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.469867 17636 leader_election.cc:288] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:42.472365 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.470075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:44007 (local address 127.14.200.1:43517)
0810 20:59:42.470108 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.470110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.470119 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:42.470145 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.470145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.470154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.470222 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.470361 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.470821 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.471660 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.472006 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.472058 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.472063 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.472269 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.472308 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 20:59:42.472381 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.470066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:51573 (local address 127.14.200.2:43311)
0810 20:59:42.470102 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.470104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.470236 (+ 132us) server_negotiation.cc:407] Connection header received
0810 20:59:42.470281 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.470282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.470290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.470342 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.470474 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.470867 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.471688 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.472060 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.472136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.472144 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.472288 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.472326 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 20:59:42.472389 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.470005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.14.200.3:44007)
0810 20:59:42.470053 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.470063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.470100 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.470236 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.470237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.470239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.470298 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.470300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.470893 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.470894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.471579 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.471581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.472098 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.472100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.472188 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.472195 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.472251 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.472287 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":23}
I20260810 20:59:42.472400 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.470005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43311 (local address 127.14.200.3:51573)
0810 20:59:42.470168 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.470178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.470230 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.470351 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.470352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.470353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.470414 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.470416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.470950 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.470951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.471589 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.471591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.472152 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.472153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.472207 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.472210 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.472268 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.472305 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":21,"thread_start_us":70,"threads_started":1}
I20260810 20:59:42.472675 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388b5b7995564dc0b8dbd1f3c7289e4e" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" is_pre_election: true
I20260810 20:59:42.472827 17250 raft_consensus.cc:2454] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:42.472756 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388b5b7995564dc0b8dbd1f3c7289e4e" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96" is_pre_election: true
I20260810 20:59:42.472908 17402 raft_consensus.cc:2454] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:42.473014 17480 leader_election.cc:302] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3ebaba173347a7bb1cf54a7f69a8b2, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:42.473143 17636 raft_consensus.cc:2781] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:42.473196 17636 raft_consensus.cc:482] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:42.473237 17636 raft_consensus.cc:3037] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.473810 17636 raft_consensus.cc:504] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.473943 17636 leader_election.cc:288] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:42.474095 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388b5b7995564dc0b8dbd1f3c7289e4e" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96"
I20260810 20:59:42.474181 17402 raft_consensus.cc:3037] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.474200 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388b5b7995564dc0b8dbd1f3c7289e4e" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
I20260810 20:59:42.474268 17250 raft_consensus.cc:3037] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.474785 17402 raft_consensus.cc:2454] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:42.474952 17250 raft_consensus.cc:2454] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:42.474948 17481 leader_election.cc:302] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:42.475052 17636 raft_consensus.cc:2781] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:42.475189 17636 raft_consensus.cc:686] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Becoming Leader. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Running, Role: LEADER
I20260810 20:59:42.475292 17636 consensus_queue.cc:237] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } }
I20260810 20:59:42.475936 17077 catalog_manager.cc:5495] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } health_report { overall_health: UNKNOWN } } }
W20260810 20:59:42.494899 17454 tablet.cc:2369] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 20:59:42 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395581, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:42.499348 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.493426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45667 (local address 127.0.0.1:43362)
0810 20:59:42.493566 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.493575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.493629 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.493726 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.493727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.493982 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:42.494031 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.494033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.494601 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.494601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.495109 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.495111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.495401 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.495401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.495435 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.495859 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:42.495863 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:42.497183 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:42.498443 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.498450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.498451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.498647 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.498849 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:42.498850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:42.498850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:42.498904 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:42.499095 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:42.499101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:42.499136 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.499194 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.499274 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 20:59:42.499395 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.493495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.14.200.62:45667)
0810 20:59:42.493564 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.493565 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.493631 (+ 66us) server_negotiation.cc:407] Connection header received
0810 20:59:42.493666 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.493667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.493675 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.493717 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:42.494089 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.494538 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.495174 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.495343 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.495390 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.497254 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:42.497256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:42.497256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:42.497261 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:42.498380 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.498749 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.498750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.498750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.498795 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:42.498958 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:42.498959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:42.498959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:42.499045 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:42.499090 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.499212 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.499314 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":46}
I20260810 20:59:42.501093 17077 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:43362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 20:59:42.501264 17077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:42.502516 17380 tablet_service.cc:1467] Processing CreateTablet for tablet 77686f9a8f5649d599a158de429f21ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b1f20da512c4f2dbd821681cbe2f429]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:42.502550 17229 tablet_service.cc:1467] Processing CreateTablet for tablet 77686f9a8f5649d599a158de429f21ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b1f20da512c4f2dbd821681cbe2f429]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:42.502668 17229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77686f9a8f5649d599a158de429f21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.502668 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77686f9a8f5649d599a158de429f21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.502658 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 77686f9a8f5649d599a158de429f21ce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b1f20da512c4f2dbd821681cbe2f429]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 20:59:42.502763 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77686f9a8f5649d599a158de429f21ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.503729 17629 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: Bootstrap starting.
I20260810 20:59:42.504097 17628 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: Bootstrap starting.
I20260810 20:59:42.504098 17631 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: Bootstrap starting.
I20260810 20:59:42.504117 17629 tablet_bootstrap.cc:654] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.504765 17631 tablet_bootstrap.cc:654] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.504822 17628 tablet_bootstrap.cc:654] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.505234 17629 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: No bootstrap required, opened a new log
I20260810 20:59:42.505308 17629 ts_tablet_manager.cc:1397] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:42.505468 17628 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: No bootstrap required, opened a new log
I20260810 20:59:42.505519 17631 tablet_bootstrap.cc:492] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: No bootstrap required, opened a new log
I20260810 20:59:42.505487 17629 raft_consensus.cc:348] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505533 17628 ts_tablet_manager.cc:1397] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:42.505554 17629 raft_consensus.cc:374] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.505565 17631 ts_tablet_manager.cc:1397] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:42.505587 17629 raft_consensus.cc:729] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.505635 17629 consensus_queue.cc:260] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505721 17629 ts_tablet_manager.cc:1428] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.505704 17631 raft_consensus.cc:348] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505690 17628 raft_consensus.cc:348] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505764 17631 raft_consensus.cc:374] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.505771 17628 raft_consensus.cc:374] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.505789 17631 raft_consensus.cc:729] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.505802 17628 raft_consensus.cc:729] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.505841 17628 consensus_queue.cc:260] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505859 17631 consensus_queue.cc:260] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.505946 17631 ts_tablet_manager.cc:1428] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.505947 17628 ts_tablet_manager.cc:1428] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:42.505997 17633 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.506006 17634 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.506067 17633 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.506043 17636 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.506084 17634 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.506111 17636 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.528151 17634 raft_consensus.cc:482] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:42.528229 17634 raft_consensus.cc:504] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.528604 17634 leader_election.cc:288] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311), ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171)
I20260810 20:59:42.531785 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.528855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:38535 (local address 127.14.200.2:43311)
0810 20:59:42.528953 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.528955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.528962 (+ 7us) server_negotiation.cc:407] Connection header received
0810 20:59:42.528991 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.528992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.528999 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.529060 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.529225 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.529688 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.530862 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.531433 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.531515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.531525 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.531688 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.531728 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 20:59:42.531778 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.528753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43311 (local address 127.14.200.1:38535)
0810 20:59:42.528811 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.528822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.528871 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.529079 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.529080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.529082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.529151 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.529152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.530014 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.530015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.530757 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.530759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.531513 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.531514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.531607 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.531610 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.531670 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.531711 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 20:59:42.531908 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.528909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:36953 (local address 127.14.200.3:39171)
0810 20:59:42.528976 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.528979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.529135 (+ 156us) server_negotiation.cc:407] Connection header received
0810 20:59:42.529214 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.529215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.529224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.529276 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.529419 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.530128 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.531047 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.531530 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.531612 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.531622 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.531786 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.531820 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 20:59:42.531997 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77686f9a8f5649d599a158de429f21ce" candidate_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96" is_pre_election: true
I20260810 20:59:42.532051 17652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.528885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39171 (local address 127.14.200.1:36953)
0810 20:59:42.529071 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.529079 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.529151 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.529283 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.529284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.529285 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.529354 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.529355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.530217 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.530219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.530942 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.530944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.531633 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.531635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.531701 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.531704 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.531776 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.531814 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":129,"threads_started":1}
I20260810 20:59:42.532084 17402 raft_consensus.cc:2454] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3ebaba173347a7bb1cf54a7f69a8b2 in term 0.
I20260810 20:59:42.532326 17176 leader_election.cc:302] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, 4b3ebaba173347a7bb1cf54a7f69a8b2; no voters:
I20260810 20:59:42.532373 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77686f9a8f5649d599a158de429f21ce" candidate_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" is_pre_election: true
I20260810 20:59:42.532476 17634 raft_consensus.cc:2781] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:42.532512 17634 raft_consensus.cc:482] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:42.532496 17556 raft_consensus.cc:2454] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b3ebaba173347a7bb1cf54a7f69a8b2 in term 0.
I20260810 20:59:42.532541 17634 raft_consensus.cc:3037] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.533265 17634 raft_consensus.cc:504] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.533399 17634 leader_election.cc:288] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311), ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171)
I20260810 20:59:42.533571 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77686f9a8f5649d599a158de429f21ce" candidate_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3"
I20260810 20:59:42.533612 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77686f9a8f5649d599a158de429f21ce" candidate_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96"
I20260810 20:59:42.533667 17556 raft_consensus.cc:3037] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.533679 17402 raft_consensus.cc:3037] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.534313 17402 raft_consensus.cc:2454] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3ebaba173347a7bb1cf54a7f69a8b2 in term 1.
I20260810 20:59:42.534310 17556 raft_consensus.cc:2454] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b3ebaba173347a7bb1cf54a7f69a8b2 in term 1.
I20260810 20:59:42.534530 17176 leader_election.cc:302] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, 4b3ebaba173347a7bb1cf54a7f69a8b2; no voters:
I20260810 20:59:42.534632 17634 raft_consensus.cc:2781] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:42.534802 17634 raft_consensus.cc:686] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Running, Role: LEADER
I20260810 20:59:42.534907 17634 consensus_queue.cc:237] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.535408 17653 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: TxnStatusTablet state changed. Reason: New leader 4b3ebaba173347a7bb1cf54a7f69a8b2. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.535501 17653 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:42.535710 17655 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 20:59:42.535701 17079 catalog_manager.cc:5495] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1). New cstate: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } health_report { overall_health: UNKNOWN } } }
I20260810 20:59:42.535836 17655 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 20:59:42.543957 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.541337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.14.200.1:43517)
0810 20:59:42.541392 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.541393 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.541402 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:42.541500 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.541500 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.541515 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.541588 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.541701 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.542131 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.543197 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.543405 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.543458 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.543676 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.543756 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.543811 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.543900 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.543917 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 20:59:42.544018 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.541268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.0.0.1:57776)
0810 20:59:42.541365 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.541375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.541420 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.541594 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.541594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.541595 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:42.541640 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.541642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.542213 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.542214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.543124 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.543127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.543484 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.543485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.543557 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.543586 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.543824 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.543832 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.543900 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.543947 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 20:59:42.545703 17556 raft_consensus.cc:1261] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Refusing update from remote peer 4b3ebaba173347a7bb1cf54a7f69a8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.545743 17402 raft_consensus.cc:1261] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Refusing update from remote peer 4b3ebaba173347a7bb1cf54a7f69a8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.545984 17634 consensus_queue.cc:1035] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.546099 17653 consensus_queue.cc:1035] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.547089 17636 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: TxnStatusTablet state changed. Reason: New leader 4b3ebaba173347a7bb1cf54a7f69a8b2. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.547188 17636 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.547335 17633 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: TxnStatusTablet state changed. Reason: New leader 4b3ebaba173347a7bb1cf54a7f69a8b2. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.547416 17633 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.548202 17634 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.548278 17634 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:42.548346 17633 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.548367 17658 mvcc.cc:205] Tried to move back new op lower bound from 7317076306104897536 to 7317076306063634432. Current Snapshot: MvccSnapshot[applied={T|T < 7317076306104897536}]
I20260810 20:59:42.548424 17633 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 0cde8918cf1046409d0605c06a6a3b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.548765 17653 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.548856 17653 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P 4b3ebaba173347a7bb1cf54a7f69a8b2: This TxnStatusTablet replica's current role is: LEADER
I20260810 20:59:42.549366 17660 mvcc.cc:205] Tried to move back new op lower bound from 7317076306104897536 to 7317076306063634432. Current Snapshot: MvccSnapshot[applied={T|T < 7317076306104897536}]
I20260810 20:59:42.549649 17636 tablet_replica.cc:439] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } } }
I20260810 20:59:42.549731 17636 tablet_replica.cc:442] T 77686f9a8f5649d599a158de429f21ce P ef19774b3e034ef18b4cc3f37c2d4fd3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 20:59:42.553776 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.551312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.14.200.3:39171)
0810 20:59:42.551345 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.551346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.551353 (+ 7us) server_negotiation.cc:407] Connection header received
0810 20:59:42.551421 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.551422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.551437 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.551508 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.551580 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.552225 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.552972 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.553276 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.553347 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.553481 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.553591 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.553666 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.553689 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.553708 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 20:59:42.558163 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.555999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:46433 (local address 127.14.200.1:43517)
0810 20:59:42.556032 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.556033 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.556041 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:42.556097 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.556098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.556106 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.556163 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.556295 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.556752 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.557492 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.557671 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.557755 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.557948 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.557979 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.558035 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.558092 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.558107 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 20:59:42.558193 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.555930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.14.200.3:46433)
0810 20:59:42.555999 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.556009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.556052 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.556179 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.556180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.556181 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:42.556229 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.556230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.556870 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.556871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.557410 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.557411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.557768 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.557769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.557847 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.557860 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.558038 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.558040 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.558088 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.558129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 20:59:42.560421 17402 raft_consensus.cc:1261] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.560503 17250 raft_consensus.cc:1261] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.560750 17636 consensus_queue.cc:1035] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.560916 17639 consensus_queue.cc:1035] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.562526 17659 mvcc.cc:205] Tried to move back new op lower bound from 7317076306166026240 to 7317076305819410432. Current Snapshot: MvccSnapshot[applied={T|T < 7317076306166026240}]
I20260810 20:59:42.569592 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.566971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:44801 (local address 127.14.200.3:39171)
0810 20:59:42.567025 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.567027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.567034 (+ 7us) server_negotiation.cc:407] Connection header received
0810 20:59:42.567054 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.567055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.567065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.567123 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.567275 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.567759 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.568857 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.569130 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.569232 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.569423 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.569447 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.569477 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.569537 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.569553 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 20:59:42.569639 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.566777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39171 (local address 127.14.200.1:44801)
0810 20:59:42.566852 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.566862 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.566899 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.567127 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.567128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.567129 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:42.567174 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.567175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.567844 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.567845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.568769 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.568793 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.569237 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.569238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.569302 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.569317 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.569487 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.569488 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.569533 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.569571 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 20:59:42.576414 17079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:42.576508 17079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:42.577344 17079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Sending DeleteTablet for 3 replicas of tablet 388b5b7995564dc0b8dbd1f3c7289e4e
I20260810 20:59:42.577693 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:42 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51258
I20260810 20:59:42.577775 17380 tablet_service.cc:1514] Processing DeleteTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:42 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:35556
I20260810 20:59:42.578173 17229 tablet_service.cc:1514] Processing DeleteTablet for tablet 388b5b7995564dc0b8dbd1f3c7289e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:42 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:57768
I20260810 20:59:42.578523 17683 tablet_replica.cc:330] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: stopping tablet replica
I20260810 20:59:42.578635 17683 raft_consensus.cc:2229] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:42.578724 17683 raft_consensus.cc:2258] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:42.578666 17079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:42.578791 17079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:42.579349 17683 ts_tablet_manager.cc:1905] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:42.579964 17380 tablet_service.cc:1467] Processing CreateTablet for tablet 27aeac977e944381a3c2d990de41cc89 (DEFAULT_TABLE table=test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.580090 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27aeac977e944381a3c2d990de41cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.581256 17683 ts_tablet_manager.cc:1918] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:42.581339 17683 log.cc:1196] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/388b5b7995564dc0b8dbd1f3c7289e4e
I20260810 20:59:42.581584 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 27aeac977e944381a3c2d990de41cc89 (DEFAULT_TABLE table=test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.581665 17683 ts_tablet_manager.cc:1939] T 388b5b7995564dc0b8dbd1f3c7289e4e P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting consensus metadata
I20260810 20:59:42.581700 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27aeac977e944381a3c2d990de41cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.582973 17628 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Bootstrap starting.
I20260810 20:59:42.582970 17629 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Bootstrap starting.
I20260810 20:59:42.583410 17628 tablet_bootstrap.cc:654] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.583428 17629 tablet_bootstrap.cc:654] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.584034 17629 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: No bootstrap required, opened a new log
I20260810 20:59:42.584053 17060 catalog_manager.cc:4841] TS 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517): tablet 388b5b7995564dc0b8dbd1f3c7289e4e (table test-table [id=f731fcfe0d7340ab92821b3e575b56ba]) successfully deleted
I20260810 20:59:42.584088 17629 ts_tablet_manager.cc:1397] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:42.584244 17629 raft_consensus.cc:348] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.584314 17629 raft_consensus.cc:374] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.584340 17629 raft_consensus.cc:729] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.584328 17229 tablet_service.cc:1467] Processing CreateTablet for tablet 27aeac977e944381a3c2d990de41cc89 (DEFAULT_TABLE table=test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:42.584380 17629 consensus_queue.cc:260] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.584437 17229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27aeac977e944381a3c2d990de41cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:42.584492 17629 ts_tablet_manager.cc:1428] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:42.584872 17682 tablet_replica.cc:330] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: stopping tablet replica
I20260810 20:59:42.584996 17682 raft_consensus.cc:2229] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:42.585055 17682 raft_consensus.cc:2258] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:42.585427 17681 tablet_replica.cc:330] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: stopping tablet replica
I20260810 20:59:42.585522 17681 raft_consensus.cc:2229] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:42.585553 17682 ts_tablet_manager.cc:1905] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:42.585626 17681 raft_consensus.cc:2258] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:42.585668 17631 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Bootstrap starting.
I20260810 20:59:42.586445 17628 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: No bootstrap required, opened a new log
I20260810 20:59:42.586494 17628 ts_tablet_manager.cc:1397] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 20:59:42.586550 17681 ts_tablet_manager.cc:1905] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:42.586627 17628 raft_consensus.cc:348] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.586694 17628 raft_consensus.cc:374] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.586716 17628 raft_consensus.cc:729] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.586756 17628 consensus_queue.cc:260] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.586836 17628 ts_tablet_manager.cc:1428] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:42.586858 17682 ts_tablet_manager.cc:1918] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:42.586959 17682 log.cc:1196] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/388b5b7995564dc0b8dbd1f3c7289e4e
I20260810 20:59:42.587100 17631 tablet_bootstrap.cc:654] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:42.587204 17682 ts_tablet_manager.cc:1939] T 388b5b7995564dc0b8dbd1f3c7289e4e P 0cde8918cf1046409d0605c06a6a3b96: Deleting consensus metadata
I20260810 20:59:42.587682 17061 catalog_manager.cc:4841] TS 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311): tablet 388b5b7995564dc0b8dbd1f3c7289e4e (table test-table [id=f731fcfe0d7340ab92821b3e575b56ba]) successfully deleted
I20260810 20:59:42.589313 17681 ts_tablet_manager.cc:1918] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:42.589404 17681 log.cc:1196] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/388b5b7995564dc0b8dbd1f3c7289e4e
I20260810 20:59:42.589623 17681 ts_tablet_manager.cc:1939] T 388b5b7995564dc0b8dbd1f3c7289e4e P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting consensus metadata
I20260810 20:59:42.589988 17064 catalog_manager.cc:4841] TS ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171): tablet 388b5b7995564dc0b8dbd1f3c7289e4e (table test-table [id=f731fcfe0d7340ab92821b3e575b56ba]) successfully deleted
I20260810 20:59:42.590178 17631 tablet_bootstrap.cc:492] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: No bootstrap required, opened a new log
I20260810 20:59:42.590227 17631 ts_tablet_manager.cc:1397] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260810 20:59:42.590349 17631 raft_consensus.cc:348] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.590440 17631 raft_consensus.cc:374] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:42.590458 17631 raft_consensus.cc:729] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Initialized, Role: FOLLOWER
I20260810 20:59:42.590497 17631 consensus_queue.cc:260] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.590575 17631 ts_tablet_manager.cc:1428] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260810 20:59:42.596336 17299 tablet.cc:2369] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 20:59:42.658517 17608 tablet.cc:2369] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 20:59:42.892833 17633 raft_consensus.cc:482] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:42.892942 17633 raft_consensus.cc:504] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.893356 17633 leader_election.cc:288] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171)
I20260810 20:59:42.897259 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.893613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39289 (local address 127.14.200.3:39171)
0810 20:59:42.893664 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.893666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.893759 (+ 93us) server_negotiation.cc:407] Connection header received
0810 20:59:42.893807 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.893808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.893818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.893886 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.894099 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.894845 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.896312 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.896885 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.896966 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.896973 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.897148 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.897187 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 20:59:42.897294 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.893540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39171 (local address 127.14.200.2:39289)
0810 20:59:42.893670 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.893681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.893752 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.893902 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.893903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.893905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.894024 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.894025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.894932 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.894933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.896218 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.896223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.896972 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.896973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.897070 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.897075 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.897121 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.897183 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":120,"thread_start_us":47,"threads_started":1}
I20260810 20:59:42.897295 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.893614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:36645 (local address 127.14.200.1:43517)
0810 20:59:42.893664 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.893666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.893759 (+ 93us) server_negotiation.cc:407] Connection header received
0810 20:59:42.893812 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.893813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.893822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.893886 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:42.894099 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.894849 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.896311 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.896901 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.896964 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.896973 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.897185 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.897234 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 20:59:42.897508 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27aeac977e944381a3c2d990de41cc89" candidate_uuid: "0cde8918cf1046409d0605c06a6a3b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" is_pre_election: true
I20260810 20:59:42.897583 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.893540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.14.200.2:36645)
0810 20:59:42.893670 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.893681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.893752 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.893902 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.893903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.893908 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:42.894015 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.894017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.894933 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.894933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.896218 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.896223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.897013 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.897015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.897087 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.897091 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.897164 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.897209 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":97,"thread_start_us":64,"threads_started":1}
I20260810 20:59:42.897614 17556 raft_consensus.cc:2454] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cde8918cf1046409d0605c06a6a3b96 in term 0.
I20260810 20:59:42.897811 17330 leader_election.cc:302] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:42.897836 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27aeac977e944381a3c2d990de41cc89" candidate_uuid: "0cde8918cf1046409d0605c06a6a3b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" is_pre_election: true
I20260810 20:59:42.897933 17250 raft_consensus.cc:2454] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cde8918cf1046409d0605c06a6a3b96 in term 0.
I20260810 20:59:42.897976 17633 raft_consensus.cc:2781] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:42.898021 17633 raft_consensus.cc:482] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:42.898049 17633 raft_consensus.cc:3037] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.898764 17633 raft_consensus.cc:504] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.898906 17633 leader_election.cc:288] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171)
I20260810 20:59:42.899065 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27aeac977e944381a3c2d990de41cc89" candidate_uuid: "0cde8918cf1046409d0605c06a6a3b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
I20260810 20:59:42.899065 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27aeac977e944381a3c2d990de41cc89" candidate_uuid: "0cde8918cf1046409d0605c06a6a3b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3"
I20260810 20:59:42.899155 17556 raft_consensus.cc:3037] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.899142 17250 raft_consensus.cc:3037] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:42.899848 17250 raft_consensus.cc:2454] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cde8918cf1046409d0605c06a6a3b96 in term 1.
I20260810 20:59:42.900000 17556 raft_consensus.cc:2454] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cde8918cf1046409d0605c06a6a3b96 in term 1.
I20260810 20:59:42.900038 17327 leader_election.cc:302] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, 4b3ebaba173347a7bb1cf54a7f69a8b2; no voters:
I20260810 20:59:42.900131 17633 raft_consensus.cc:2781] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:42.900245 17633 raft_consensus.cc:686] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 LEADER]: Becoming Leader. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Running, Role: LEADER
I20260810 20:59:42.900318 17633 consensus_queue.cc:237] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:42.901007 17076 catalog_manager.cc:5495] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "0cde8918cf1046409d0605c06a6a3b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } health_report { overall_health: UNKNOWN } } }
I20260810 20:59:42.946717 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.944242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.14.200.2:43311)
0810 20:59:42.944283 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.944284 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.944292 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:42.944361 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.944361 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.944383 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.944466 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.944622 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.945124 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.945869 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.946086 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.946162 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.946321 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.946438 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.946533 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.946581 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.946614 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 20:59:42.951611 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.949129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50591 (local address 127.14.200.1:43517)
0810 20:59:42.949193 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.949195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.949289 (+ 94us) server_negotiation.cc:407] Connection header received
0810 20:59:42.949367 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.949368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.949381 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.949425 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.949607 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.950031 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.950845 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.951049 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.951121 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.951292 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.951353 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.951433 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.951514 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.951539 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 20:59:42.951941 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.949047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.14.200.2:50591)
0810 20:59:42.949229 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.949237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.949303 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.949426 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.949426 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.949428 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:42.949541 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.949543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.950113 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.950113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.950760 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.950762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.951121 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.951122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.951198 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.951211 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.951435 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.951438 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.951509 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.951562 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260810 20:59:42.954137 17556 raft_consensus.cc:1261] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Refusing update from remote peer 0cde8918cf1046409d0605c06a6a3b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.954196 17250 raft_consensus.cc:1261] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Refusing update from remote peer 0cde8918cf1046409d0605c06a6a3b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:42.954483 17689 consensus_queue.cc:1035] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.954634 17633 consensus_queue.cc:1035] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:42.961478 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.959384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:54519 (local address 127.14.200.2:43311)
0810 20:59:42.959435 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:42.959436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:42.959465 (+ 29us) server_negotiation.cc:407] Connection header received
0810 20:59:42.959547 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:42.959548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:42.959558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:42.959626 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:42.959794 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.960245 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.960848 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.961024 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.961071 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.961237 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.961268 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.961316 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:42.961390 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 20:59:42.961411 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260810 20:59:42.961508 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:42.959356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43311 (local address 127.14.200.1:54519)
0810 20:59:42.959400 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:42.959427 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:42.959479 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:42.959639 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:42.959640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:42.959641 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 20:59:42.959696 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.959697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.960307 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.960308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.960793 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:42.960794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:42.961100 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:42.961101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:42.961153 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:42.961158 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 20:59:42.961339 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 20:59:42.961342 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:42.961389 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 20:59:42.961437 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 20:59:42.961769 17660 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 20:59:44.481572 17076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:44.481693 17076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:44.482551 17076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Sending DeleteTablet for 3 replicas of tablet 27aeac977e944381a3c2d990de41cc89
I20260810 20:59:44.482841 17380 tablet_service.cc:1514] Processing DeleteTablet for tablet 27aeac977e944381a3c2d990de41cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:35556
I20260810 20:59:44.482894 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 27aeac977e944381a3c2d990de41cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51258
I20260810 20:59:44.483126 17229 tablet_service.cc:1514] Processing DeleteTablet for tablet 27aeac977e944381a3c2d990de41cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:57768
I20260810 20:59:44.484712 17725 tablet_replica.cc:330] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: stopping tablet replica
I20260810 20:59:44.484877 17726 tablet_replica.cc:330] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: stopping tablet replica
I20260810 20:59:44.484979 17726 raft_consensus.cc:2229] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:44.485096 17725 raft_consensus.cc:2229] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:44.485185 17726 raft_consensus.cc:2258] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.485318 17725 raft_consensus.cc:2258] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.485657 17726 ts_tablet_manager.cc:1905] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:44.485958 17725 ts_tablet_manager.cc:1905] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:44.487802 17726 ts_tablet_manager.cc:1918] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 20:59:44.487869 17726 log.cc:1196] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/27aeac977e944381a3c2d990de41cc89
I20260810 20:59:44.488050 17727 tablet_replica.cc:330] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: stopping tablet replica
I20260810 20:59:44.488117 17726 ts_tablet_manager.cc:1939] T 27aeac977e944381a3c2d990de41cc89 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting consensus metadata
I20260810 20:59:44.488215 17727 raft_consensus.cc:2229] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:44.488308 17727 raft_consensus.cc:2258] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.488493 17064 catalog_manager.cc:4841] TS ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171): tablet 27aeac977e944381a3c2d990de41cc89 (table test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]) successfully deleted
I20260810 20:59:44.488607 17727 ts_tablet_manager.cc:1905] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:44.488883 17725 ts_tablet_manager.cc:1918] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 20:59:44.488946 17725 log.cc:1196] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/27aeac977e944381a3c2d990de41cc89
I20260810 20:59:44.489262 17725 ts_tablet_manager.cc:1939] T 27aeac977e944381a3c2d990de41cc89 P 0cde8918cf1046409d0605c06a6a3b96: Deleting consensus metadata
I20260810 20:59:44.489653 17061 catalog_manager.cc:4841] TS 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311): tablet 27aeac977e944381a3c2d990de41cc89 (table test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]) successfully deleted
I20260810 20:59:44.490244 17727 ts_tablet_manager.cc:1918] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 20:59:44.490300 17727 log.cc:1196] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/27aeac977e944381a3c2d990de41cc89
I20260810 20:59:44.490545 17727 ts_tablet_manager.cc:1939] T 27aeac977e944381a3c2d990de41cc89 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting consensus metadata
I20260810 20:59:44.491041 17060 catalog_manager.cc:4841] TS 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517): tablet 27aeac977e944381a3c2d990de41cc89 (table test-table [id=be2f07a0c97c47b2b96bad6e38d61e1c]) successfully deleted
Aug 10 20:59:44 dist-test-slave-1x4g krb5kdc[17031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395584, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 20:59:44 dist-test-slave-1x4g krb5kdc[17031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395584, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:44.507870 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:44.500915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.14.200.62:45667)
0810 20:59:44.501056 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:44.501058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:44.501097 (+ 39us) server_negotiation.cc:407] Connection header received
0810 20:59:44.501169 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:44.501170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:44.501180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:44.501245 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:44.501769 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:44.502253 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:44.502939 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:44.503130 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:44.503186 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:44.505442 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:44.505444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:44.505444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:44.505450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:44.506973 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:44.507251 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:44.507251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:44.507252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:44.507306 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:44.507475 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:44.507475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:44.507476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:44.507571 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:44.507646 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:44.507724 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 20:59:44.507788 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260810 20:59:44.509375 17076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:44.509555 17076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:44.510962 17229 tablet_service.cc:1467] Processing CreateTablet for tablet c84dfee654554be8b99317d23854a9b0 (DEFAULT_TABLE table=test-table [id=ae51a4d9ccca4fc384475e68833105c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.511022 17380 tablet_service.cc:1467] Processing CreateTablet for tablet c84dfee654554be8b99317d23854a9b0 (DEFAULT_TABLE table=test-table [id=ae51a4d9ccca4fc384475e68833105c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.511112 17229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84dfee654554be8b99317d23854a9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.511089 17535 tablet_service.cc:1467] Processing CreateTablet for tablet c84dfee654554be8b99317d23854a9b0 (DEFAULT_TABLE table=test-table [id=ae51a4d9ccca4fc384475e68833105c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.511154 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84dfee654554be8b99317d23854a9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.511199 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84dfee654554be8b99317d23854a9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.512331 17736 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Bootstrap starting.
I20260810 20:59:44.512595 17737 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Bootstrap starting.
I20260810 20:59:44.512869 17736 tablet_bootstrap.cc:654] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.513013 17737 tablet_bootstrap.cc:654] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.513391 17738 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Bootstrap starting.
I20260810 20:59:44.513651 17736 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: No bootstrap required, opened a new log
I20260810 20:59:44.513736 17736 ts_tablet_manager.cc:1397] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:44.513640 17737 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: No bootstrap required, opened a new log
I20260810 20:59:44.513867 17737 ts_tablet_manager.cc:1397] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:44.513870 17736 raft_consensus.cc:348] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.513955 17736 raft_consensus.cc:374] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.513989 17736 raft_consensus.cc:729] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.513988 17737 raft_consensus.cc:348] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.514055 17736 consensus_queue.cc:260] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.514096 17737 raft_consensus.cc:374] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.514120 17737 raft_consensus.cc:729] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.514191 17737 consensus_queue.cc:260] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.514205 17736 ts_tablet_manager.cc:1428] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:44.514290 17737 ts_tablet_manager.cc:1428] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:44.514670 17738 tablet_bootstrap.cc:654] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.515385 17738 tablet_bootstrap.cc:492] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: No bootstrap required, opened a new log
I20260810 20:59:44.515451 17738 ts_tablet_manager.cc:1397] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:44.515568 17738 raft_consensus.cc:348] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.515638 17738 raft_consensus.cc:374] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.515671 17738 raft_consensus.cc:729] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.515726 17738 consensus_queue.cc:260] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.515883 17738 ts_tablet_manager.cc:1428] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:44.524961 17743 raft_consensus.cc:482] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:44.525027 17743 raft_consensus.cc:504] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.525156 17743 leader_election.cc:288] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:44.525295 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84dfee654554be8b99317d23854a9b0" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" is_pre_election: true
I20260810 20:59:44.525316 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84dfee654554be8b99317d23854a9b0" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96" is_pre_election: true
I20260810 20:59:44.525388 17250 raft_consensus.cc:2454] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:44.525404 17402 raft_consensus.cc:2454] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:44.525542 17480 leader_election.cc:302] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3ebaba173347a7bb1cf54a7f69a8b2, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:44.525636 17743 raft_consensus.cc:2781] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:44.525674 17743 raft_consensus.cc:482] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:44.525703 17743 raft_consensus.cc:3037] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.526353 17743 raft_consensus.cc:504] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.526502 17743 leader_election.cc:288] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:44.526684 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84dfee654554be8b99317d23854a9b0" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96"
I20260810 20:59:44.526711 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84dfee654554be8b99317d23854a9b0" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
I20260810 20:59:44.526762 17402 raft_consensus.cc:3037] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.526793 17250 raft_consensus.cc:3037] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.527341 17250 raft_consensus.cc:2454] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:44.527407 17402 raft_consensus.cc:2454] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:44.527508 17480 leader_election.cc:302] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3ebaba173347a7bb1cf54a7f69a8b2, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:44.527601 17743 raft_consensus.cc:2781] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:44.527772 17743 raft_consensus.cc:686] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Becoming Leader. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Running, Role: LEADER
I20260810 20:59:44.527837 17743 consensus_queue.cc:237] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.528353 17076 catalog_manager.cc:5495] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } health_report { overall_health: HEALTHY } } }
I20260810 20:59:44.538551 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:44.536462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.14.200.3:39171)
0810 20:59:44.536560 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:44.536561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:44.536566 (+ 5us) server_negotiation.cc:407] Connection header received
0810 20:59:44.536583 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:44.536584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:44.536602 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:44.536670 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:44.536909 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:44.537349 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:44.538053 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:44.538216 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:44.538261 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:44.538352 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:44.538375 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:44.538404 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:44.538451 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 20:59:44.538464 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":61,"threads_started":1}
I20260810 20:59:44.540571 17250 raft_consensus.cc:1261] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:44.540586 17402 raft_consensus.cc:1261] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:44.540797 17743 consensus_queue.cc:1035] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:44.540899 17744 consensus_queue.cc:1035] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:44.550419 17077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:44.550490 17077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:44.551110 17077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Sending DeleteTablet for 3 replicas of tablet c84dfee654554be8b99317d23854a9b0
I20260810 20:59:44.551327 17229 tablet_service.cc:1514] Processing DeleteTablet for tablet c84dfee654554be8b99317d23854a9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:57768
I20260810 20:59:44.551347 17380 tablet_service.cc:1514] Processing DeleteTablet for tablet c84dfee654554be8b99317d23854a9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:35556
I20260810 20:59:44.551394 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet c84dfee654554be8b99317d23854a9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:44 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51258
I20260810 20:59:44.551445 17725 tablet_replica.cc:330] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: stopping tablet replica
I20260810 20:59:44.551471 17726 tablet_replica.cc:330] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: stopping tablet replica
I20260810 20:59:44.551477 17727 tablet_replica.cc:330] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: stopping tablet replica
I20260810 20:59:44.551505 17725 raft_consensus.cc:2229] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:44.551522 17726 raft_consensus.cc:2229] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:44.551532 17727 raft_consensus.cc:2229] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:44.551551 17725 raft_consensus.cc:2258] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.551569 17727 raft_consensus.cc:2258] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.551579 17726 raft_consensus.cc:2258] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:44.551743 17077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 20:59:44.551901 17727 ts_tablet_manager.cc:1905] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:44.551923 17726 ts_tablet_manager.cc:1905] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 20:59:44.551932 17077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:44.553530 17725 ts_tablet_manager.cc:1905] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:44.553565 17726 ts_tablet_manager.cc:1918] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:44.553624 17726 log.cc:1196] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/c84dfee654554be8b99317d23854a9b0
I20260810 20:59:44.553884 17726 ts_tablet_manager.cc:1939] T c84dfee654554be8b99317d23854a9b0 P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting consensus metadata
I20260810 20:59:44.554093 17727 ts_tablet_manager.cc:1918] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:44.554141 17727 log.cc:1196] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/c84dfee654554be8b99317d23854a9b0
I20260810 20:59:44.554123 17380 tablet_service.cc:1467] Processing CreateTablet for tablet 77dc716030ff4856a71ffa6a6a25451d (DEFAULT_TABLE table=test-table [id=f55196481cd44ddaad983457adf26f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.554236 17380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dc716030ff4856a71ffa6a6a25451d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.554358 17727 ts_tablet_manager.cc:1939] T c84dfee654554be8b99317d23854a9b0 P 4b3ebaba173347a7bb1cf54a7f69a8b2: Deleting consensus metadata
I20260810 20:59:44.554675 17060 catalog_manager.cc:4841] TS 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517): tablet c84dfee654554be8b99317d23854a9b0 (table test-table [id=ae51a4d9ccca4fc384475e68833105c3]) successfully deleted
I20260810 20:59:44.555176 17725 ts_tablet_manager.cc:1918] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 20:59:44.555248 17725 log.cc:1196] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/c84dfee654554be8b99317d23854a9b0
I20260810 20:59:44.555364 17737 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Bootstrap starting.
I20260810 20:59:44.555419 17229 tablet_service.cc:1467] Processing CreateTablet for tablet 77dc716030ff4856a71ffa6a6a25451d (DEFAULT_TABLE table=test-table [id=f55196481cd44ddaad983457adf26f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.555457 17725 ts_tablet_manager.cc:1939] T c84dfee654554be8b99317d23854a9b0 P 0cde8918cf1046409d0605c06a6a3b96: Deleting consensus metadata
I20260810 20:59:44.555523 17229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dc716030ff4856a71ffa6a6a25451d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.555712 17737 tablet_bootstrap.cc:654] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.555807 17061 catalog_manager.cc:4841] TS 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311): tablet c84dfee654554be8b99317d23854a9b0 (table test-table [id=ae51a4d9ccca4fc384475e68833105c3]) successfully deleted
I20260810 20:59:44.556278 17737 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: No bootstrap required, opened a new log
I20260810 20:59:44.556342 17737 ts_tablet_manager.cc:1397] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:44.556447 17737 raft_consensus.cc:348] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.556527 17737 raft_consensus.cc:374] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.556555 17737 raft_consensus.cc:729] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cde8918cf1046409d0605c06a6a3b96, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.556587 17736 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2: Bootstrap starting.
I20260810 20:59:44.556604 17737 consensus_queue.cc:260] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.556720 17737 ts_tablet_manager.cc:1428] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:44.556807 17064 catalog_manager.cc:4841] TS ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171): tablet c84dfee654554be8b99317d23854a9b0 (table test-table [id=ae51a4d9ccca4fc384475e68833105c3]) successfully deleted
I20260810 20:59:44.557041 17736 tablet_bootstrap.cc:654] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.557040 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 77dc716030ff4856a71ffa6a6a25451d (DEFAULT_TABLE table=test-table [id=f55196481cd44ddaad983457adf26f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:44.557140 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dc716030ff4856a71ffa6a6a25451d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:44.557678 17736 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2: No bootstrap required, opened a new log
I20260810 20:59:44.557715 17736 ts_tablet_manager.cc:1397] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:44.557821 17736 raft_consensus.cc:348] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.557878 17736 raft_consensus.cc:374] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.557893 17736 raft_consensus.cc:729] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3ebaba173347a7bb1cf54a7f69a8b2, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.557928 17736 consensus_queue.cc:260] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.558025 17736 ts_tablet_manager.cc:1428] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:44.558167 17738 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Bootstrap starting.
I20260810 20:59:44.558451 17738 tablet_bootstrap.cc:654] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:44.559041 17738 tablet_bootstrap.cc:492] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: No bootstrap required, opened a new log
I20260810 20:59:44.559078 17738 ts_tablet_manager.cc:1397] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:44.559185 17738 raft_consensus.cc:348] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.559244 17738 raft_consensus.cc:374] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:44.559257 17738 raft_consensus.cc:729] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Initialized, Role: FOLLOWER
I20260810 20:59:44.559309 17738 consensus_queue.cc:260] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.559391 17738 ts_tablet_manager.cc:1428] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:44.715610 17743 raft_consensus.cc:482] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:44.715698 17743 raft_consensus.cc:504] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.715884 17743 leader_election.cc:288] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:44.716250 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dc716030ff4856a71ffa6a6a25451d" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" is_pre_election: true
I20260810 20:59:44.716250 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dc716030ff4856a71ffa6a6a25451d" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96" is_pre_election: true
I20260810 20:59:44.716375 17250 raft_consensus.cc:2454] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:44.716375 17402 raft_consensus.cc:2454] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 0.
I20260810 20:59:44.716578 17481 leader_election.cc:302] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:44.716723 17743 raft_consensus.cc:2781] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:44.716811 17743 raft_consensus.cc:482] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:44.716854 17743 raft_consensus.cc:3037] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.717559 17743 raft_consensus.cc:504] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.717720 17743 leader_election.cc:288] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Requested vote from peers 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517), 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311)
I20260810 20:59:44.717891 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dc716030ff4856a71ffa6a6a25451d" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2"
I20260810 20:59:44.717916 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dc716030ff4856a71ffa6a6a25451d" candidate_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cde8918cf1046409d0605c06a6a3b96"
I20260810 20:59:44.717993 17250 raft_consensus.cc:3037] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.718026 17402 raft_consensus.cc:3037] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:44.718643 17402 raft_consensus.cc:2454] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:44.718673 17250 raft_consensus.cc:2454] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef19774b3e034ef18b4cc3f37c2d4fd3 in term 1.
I20260810 20:59:44.718829 17481 leader_election.cc:302] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cde8918cf1046409d0605c06a6a3b96, ef19774b3e034ef18b4cc3f37c2d4fd3; no voters:
I20260810 20:59:44.718943 17743 raft_consensus.cc:2781] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:44.719017 17743 raft_consensus.cc:686] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Becoming Leader. State: Replica: ef19774b3e034ef18b4cc3f37c2d4fd3, State: Running, Role: LEADER
I20260810 20:59:44.719081 17743 consensus_queue.cc:237] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } }
I20260810 20:59:44.719705 17077 catalog_manager.cc:5495] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef19774b3e034ef18b4cc3f37c2d4fd3" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39171 } health_report { overall_health: HEALTHY } } }
I20260810 20:59:44.740633 17401 raft_consensus.cc:1261] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:44.740669 17250 raft_consensus.cc:1261] T 77dc716030ff4856a71ffa6a6a25451d P 4b3ebaba173347a7bb1cf54a7f69a8b2 [term 1 FOLLOWER]: Refusing update from remote peer ef19774b3e034ef18b4cc3f37c2d4fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:44.740955 17752 consensus_queue.cc:1035] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3ebaba173347a7bb1cf54a7f69a8b2" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:44.741074 17743 consensus_queue.cc:1035] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cde8918cf1046409d0605c06a6a3b96" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 20:59:44.745267 17705 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 20:59:46.264643 17077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:46.264750 17077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:46.265534 17077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e1d74c86f15433e9287cd823dd57ef1: Sending DeleteTablet for 3 replicas of tablet 77dc716030ff4856a71ffa6a6a25451d
I20260810 20:59:46.265805 17229 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dc716030ff4856a71ffa6a6a25451d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:46 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:57768
I20260810 20:59:46.265856 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dc716030ff4856a71ffa6a6a25451d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:46 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51258
I20260810 20:59:46.266002 17380 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dc716030ff4856a71ffa6a6a25451d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:46 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:35556
I20260810 20:59:46.266000 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17158
I20260810 20:59:46.268033 17786 tablet_replica.cc:330] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: stopping tablet replica
I20260810 20:59:46.268115 17786 raft_consensus.cc:2229] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:46.268162 17786 raft_consensus.cc:2258] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:46.268414 17786 ts_tablet_manager.cc:1905] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:46.269999 17786 ts_tablet_manager.cc:1918] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 20:59:46.270071 17786 log.cc:1196] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/77dc716030ff4856a71ffa6a6a25451d
I20260810 20:59:46.270287 17786 ts_tablet_manager.cc:1939] T 77dc716030ff4856a71ffa6a6a25451d P 0cde8918cf1046409d0605c06a6a3b96: Deleting consensus metadata
I20260810 20:59:46.270640 17061 catalog_manager.cc:4841] TS 0cde8918cf1046409d0605c06a6a3b96 (127.14.200.2:43311): tablet 77dc716030ff4856a71ffa6a6a25451d (table test-table [id=f55196481cd44ddaad983457adf26f25]) successfully deleted
I20260810 20:59:46.272960 17784 tablet_replica.cc:330] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: stopping tablet replica
I20260810 20:59:46.273020 17784 raft_consensus.cc:2229] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:46.273088 17784 raft_consensus.cc:2258] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:46.273288 17784 ts_tablet_manager.cc:1905] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:46.274485 17784 ts_tablet_manager.cc:1918] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 20:59:46.274538 17784 log.cc:1196] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/77dc716030ff4856a71ffa6a6a25451d
I20260810 20:59:46.274778 17784 ts_tablet_manager.cc:1939] T 77dc716030ff4856a71ffa6a6a25451d P ef19774b3e034ef18b4cc3f37c2d4fd3: Deleting consensus metadata
I20260810 20:59:46.275106 17064 catalog_manager.cc:4841] TS ef19774b3e034ef18b4cc3f37c2d4fd3 (127.14.200.3:39171): tablet 77dc716030ff4856a71ffa6a6a25451d (table test-table [id=f55196481cd44ddaad983457adf26f25]) successfully deleted
W20260810 20:59:46.275806 17060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:43517 (error 108)
I20260810 20:59:46.276037 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17308
I20260810 20:59:46.278587 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:46.276154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43517 (local address 127.0.0.1:57786)
0810 20:59:46.278477 (+ 2323us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:46.278526 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2300,"thread_start_us":64,"threads_started":1}
W20260810 20:59:46.278712 17060 catalog_manager.cc:4584] TS 4b3ebaba173347a7bb1cf54a7f69a8b2 (127.14.200.1:43517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77dc716030ff4856a71ffa6a6a25451d: Network error: Client connection negotiation failed: client connection to 127.14.200.1:43517: connect: Connection refused (error 111)
I20260810 20:59:46.285032 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17462
I20260810 20:59:46.291265 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17044
2026-08-10T20:59:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5657 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:46 dist-test-slave-1x4g krb5kdc[17791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:46 dist-test-slave-1x4g krb5kdc[17791](info): set up 2 sockets
Aug 10 20:59:46 dist-test-slave-1x4g krb5kdc[17791](info): commencing operation
krb5kdc: starting...
W20260810 20:59:47.315647 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:47.429406 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:32917
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:32917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:47.506485 17808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:47.506695 17808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:47.506752 17808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:47.507818 17808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:47.507886 17808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:47.507910 17808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:47.507931 17808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:47.507961 17808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:47.509567 17808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:32917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:32917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:47.509827 17808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:47.510046 17808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:47.514693 17818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:47.514722 17816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.514953 17808 server_base.cc:1034] running on GCE node
W20260810 20:59:47.514761 17814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.515213 17808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:47.515468 17808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:47.516657 17808 hybrid_clock.cc:648] HybridClock initialized: now 1786395587516621 us; error 50 us; skew 500 ppm
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:47.519102 17808 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:47.519528 17808 webserver.cc:460] Webserver started at http://127.14.200.62:35611/ using document root <none> and password file <none>
I20260810 20:59:47.519730 17808 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:47.519790 17808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:47.519928 17808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:47.520974 17808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "6801a840f33344699c9023489ff8bba7"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "76568dd7b8bdfbc1bc2faf66b7517074"
server_key_iv: "96cc7fb3bbf99da4a7d74568b72a68a0"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.521396 17808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "6801a840f33344699c9023489ff8bba7"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "76568dd7b8bdfbc1bc2faf66b7517074"
server_key_iv: "96cc7fb3bbf99da4a7d74568b72a68a0"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.523018 17808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:47.523742 17825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.523917 17808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:47.524024 17808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "6801a840f33344699c9023489ff8bba7"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "76568dd7b8bdfbc1bc2faf66b7517074"
server_key_iv: "96cc7fb3bbf99da4a7d74568b72a68a0"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.524097 17808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:47.533964 17808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:47.535656 17808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:47.535807 17808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:47.539896 17808 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:32917
I20260810 20:59:47.539917 17881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:32917 every 8 connection(s)
I20260810 20:59:47.540236 17808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:47.541059 17882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:47.543080 17882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: Bootstrap starting.
I20260810 20:59:47.543699 17882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:47.543740 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17808
I20260810 20:59:47.543826 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:47.544036 15136 external_mini_cluster.cc:1428] Setting key 5c7ca7fd9297d1eb9605854c9d7b5a5e
I20260810 20:59:47.544039 17882 log.cc:824] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:47.544888 17882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: No bootstrap required, opened a new log
I20260810 20:59:47.546777 17882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } }
I20260810 20:59:47.546893 17882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:47.546913 17882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801a840f33344699c9023489ff8bba7, State: Initialized, Role: FOLLOWER
I20260810 20:59:47.547087 17882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } }
I20260810 20:59:47.547168 17882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:47.547196 17882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:47.547240 17882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:47.547843 17882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } }
I20260810 20:59:47.547957 17882 leader_election.cc:302] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6801a840f33344699c9023489ff8bba7; no voters:
I20260810 20:59:47.548105 17882 leader_election.cc:288] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:47.548135 17887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:47.548311 17887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [term 1 LEADER]: Becoming Leader. State: Replica: 6801a840f33344699c9023489ff8bba7, State: Running, Role: LEADER
I20260810 20:59:47.548323 17882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:47.548488 17887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } }
I20260810 20:59:47.550074 17889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6801a840f33344699c9023489ff8bba7. Latest consensus state: current_term: 1 leader_uuid: "6801a840f33344699c9023489ff8bba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } } }
I20260810 20:59:47.550282 17888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6801a840f33344699c9023489ff8bba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6801a840f33344699c9023489ff8bba7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 32917 } } }
I20260810 20:59:47.550341 17889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:47.550341 17888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7 [sys.catalog]: This master's current role is: LEADER
W20260810 20:59:47.550640 17903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 20:59:47.550717 17903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 20:59:47.550798 17905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:47.550993 17905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:47.552506 17905 catalog_manager.cc:1347] Generated new cluster ID: 1c8a2f701cbd4d33808bb96657f8da33
I20260810 20:59:47.552573 17905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:47.555104 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:47.544533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.14.200.62:32917)
0810 20:59:47.544686 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:47.544696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:47.544728 (+ 32us) server_negotiation.cc:407] Connection header received
0810 20:59:47.545248 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:47.545255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:47.545297 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:47.545430 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:47.546078 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.546877 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.547943 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.548554 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.548988 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:47.551385 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:47.551389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:47.551389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:47.551401 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:47.553380 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.553790 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.553794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.553794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.553890 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.554103 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.554104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.554104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.554371 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:47.554514 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:47.554788 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 20:59:47.554907 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260810 20:59:47.565932 17905 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:47.566582 17905 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:47.571206 17905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6801a840f33344699c9023489ff8bba7: Generated new TSK 0
I20260810 20:59:47.571357 17905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:47.599164 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:47.700641 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:47.700858 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:47.700896 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:47.701965 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:47.702026 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:47.702069 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:47.703425 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:47.703691 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:47.703914 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:47.708684 17917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:47.708710 17914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.708809 17909 server_base.cc:1034] running on GCE node
W20260810 20:59:47.708840 17915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.709136 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:47.709406 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:47.710572 17909 hybrid_clock.cc:648] HybridClock initialized: now 1786395587710552 us; error 35 us; skew 500 ppm
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:47.713011 17909 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:47.713470 17909 webserver.cc:460] Webserver started at http://127.14.200.1:42533/ using document root <none> and password file <none>
I20260810 20:59:47.713702 17909 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:47.713747 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:47.713919 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:47.714973 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "5aaacb189a13462db84844d9b48796c0"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "a6b4da74ef2512efde25db2adbb8e8a1"
server_key_iv: "e4fa297e4c5ed1bbdb5d26534baacd0a"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.715304 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "5aaacb189a13462db84844d9b48796c0"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "a6b4da74ef2512efde25db2adbb8e8a1"
server_key_iv: "e4fa297e4c5ed1bbdb5d26534baacd0a"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.716568 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:47.717386 17925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.717555 17909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.717643 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "5aaacb189a13462db84844d9b48796c0"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "a6b4da74ef2512efde25db2adbb8e8a1"
server_key_iv: "e4fa297e4c5ed1bbdb5d26534baacd0a"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.717697 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:47.725875 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:47.727743 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:47.727839 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:47.727990 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:47.728205 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:47.728231 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.728250 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:47.728261 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.733739 17909 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:45895
I20260810 20:59:47.733814 18050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:45895 every 8 connection(s)
I20260810 20:59:47.734108 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 20:59:47.734619 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 17909
I20260810 20:59:47.734699 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:47.734872 15136 external_mini_cluster.cc:1428] Setting key 8c9ef05ec50f38c5f40ff100f192c28b
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
I20260810 20:59:47.744575 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:47.735384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:59973 (local address 127.14.200.62:32917)
0810 20:59:47.735431 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:47.735434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:47.735674 (+ 240us) server_negotiation.cc:407] Connection header received
0810 20:59:47.736383 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:47.736384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:47.736393 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:47.736431 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:47.737683 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.738439 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.739350 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.739571 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.739625 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:47.741921 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:47.741923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:47.741923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:47.741930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:47.743199 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.743586 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.743587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.743588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.743675 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.743941 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.743941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.743942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.744049 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:47.744120 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:47.744396 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 20:59:47.744482 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 20:59:47.744747 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:47.735377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:32917 (local address 127.14.200.1:59973)
0810 20:59:47.735555 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:47.735603 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:47.736288 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:47.736492 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:47.736495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:47.737326 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:47.737596 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:47.737602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.738539 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.738539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:47.739277 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:47.739279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.739672 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.739673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:47.739854 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:47.740289 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:47.740299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:47.741833 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:47.743294 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:47.743301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:47.743304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:47.743514 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:47.743751 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:47.743752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:47.743752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:47.743858 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:47.744125 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:47.744133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:47.744263 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 20:59:47.744362 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 20:59:47.744524 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260810 20:59:47.745218 18051 heartbeater.cc:347] Connected to a master server at 127.14.200.62:32917
I20260810 20:59:47.745291 18051 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:47.745489 18051 heartbeater.cc:511] Master 127.14.200.62:32917 requested a full tablet report, sending...
I20260810 20:59:47.746083 17845 ts_manager.cc:194] Registered new tserver with Master: 5aaacb189a13462db84844d9b48796c0 (127.14.200.1:45895)
I20260810 20:59:47.746735 17845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:59973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:47.761040 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:47.838491 18056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:47.838668 18056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:47.838701 18056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:47.839776 18056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:47.839814 18056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:47.839838 18056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:47.841168 18056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:47.841375 18056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:47.841578 18056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:47.846248 18064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:47.846279 18061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:47.846310 18062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.846477 18056 server_base.cc:1034] running on GCE node
I20260810 20:59:47.846696 18056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:47.846927 18056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:47.848063 18056 hybrid_clock.cc:648] HybridClock initialized: now 1786395587848052 us; error 27 us; skew 500 ppm
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:47.850454 18056 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:47.850889 18056 webserver.cc:460] Webserver started at http://127.14.200.2:44377/ using document root <none> and password file <none>
I20260810 20:59:47.851104 18056 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:47.851146 18056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:47.851280 18056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:47.852243 18056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "25c3ac3aa5bf4dff916a9146a1f0a4b3"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "0e69a85be69073e9d49daae8cf627acf"
server_key_iv: "46873a174aaf1019ff4f357b9545acbd"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.852596 18056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "25c3ac3aa5bf4dff916a9146a1f0a4b3"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "0e69a85be69073e9d49daae8cf627acf"
server_key_iv: "46873a174aaf1019ff4f357b9545acbd"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.854017 18056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:47.854748 18071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.854935 18056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:47.855044 18056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "25c3ac3aa5bf4dff916a9146a1f0a4b3"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "0e69a85be69073e9d49daae8cf627acf"
server_key_iv: "46873a174aaf1019ff4f357b9545acbd"
server_key_version: "encryptionkey@0"
I20260810 20:59:47.855114 18056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:47.868978 18056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:47.870668 18056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:47.870822 18056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:47.871045 18056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:47.871338 18056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:47.871387 18056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.871417 18056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:47.871441 18056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:47.877110 18056 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46127
I20260810 20:59:47.877179 18196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46127 every 8 connection(s)
I20260810 20:59:47.877457 18056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:47.885231 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18056
I20260810 20:59:47.885340 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:47.885540 15136 external_mini_cluster.cc:1428] Setting key 24438271ccba59c3feb780c2e54850e5
I20260810 20:59:47.886302 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:47.878156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:48843 (local address 127.14.200.62:32917)
0810 20:59:47.878223 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:47.878225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:47.878367 (+ 142us) server_negotiation.cc:407] Connection header received
0810 20:59:47.878863 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:47.878864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:47.878876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:47.878927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:47.879821 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.880281 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.881331 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.881522 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.881604 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:47.883569 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:47.883570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:47.883571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:47.883577 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:47.884753 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.885242 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.885243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.885244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.885306 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:47.885604 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:47.885605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:47.885606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:47.885724 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:47.885794 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:47.886087 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 20:59:47.886201 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 20:59:47.886513 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:47.878101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:32917 (local address 127.14.200.2:48843)
0810 20:59:47.878258 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:47.878297 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:47.878755 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:47.878991 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:47.878995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:47.879542 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:47.879725 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:47.879729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.880374 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.880375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:47.881232 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:47.881235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:47.881618 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:47.881619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:47.881736 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:47.882174 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:47.882182 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:47.883473 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:47.884921 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:47.884928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:47.884930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:47.885166 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:47.885389 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:47.885390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:47.885390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:47.885526 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:47.885797 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:47.885805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:47.885949 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 20:59:47.886057 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 20:59:47.886249 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260810 20:59:47.886905 18197 heartbeater.cc:347] Connected to a master server at 127.14.200.62:32917
I20260810 20:59:47.886996 18197 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:47.887161 18197 heartbeater.cc:511] Master 127.14.200.62:32917 requested a full tablet report, sending...
I20260810 20:59:47.887554 17845 ts_manager.cc:194] Registered new tserver with Master: 25c3ac3aa5bf4dff916a9146a1f0a4b3 (127.14.200.2:46127)
I20260810 20:59:47.887889 17845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:48843
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:47.909791 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:47.987879 18202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:47.988054 18202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:47.988075 18202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:47.989143 18202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:47.989190 18202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:47.989246 18202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:47.990612 18202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:32917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:47.990844 18202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:47.991060 18202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:47.995998 18210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.996176 18202 server_base.cc:1034] running on GCE node
W20260810 20:59:47.995990 18207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:47.995990 18208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:47.996464 18202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:47.996711 18202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:47.997874 18202 hybrid_clock.cc:648] HybridClock initialized: now 1786395587997873 us; error 45 us; skew 500 ppm
Aug 10 20:59:47 dist-test-slave-1x4g krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:48.000222 18202 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:48.000669 18202 webserver.cc:460] Webserver started at http://127.14.200.3:41721/ using document root <none> and password file <none>
I20260810 20:59:48.000942 18202 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:48.001006 18202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:48.001151 18202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:48.002151 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "6a7cd961adb545d1844a518fe9c5a9a4"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "7fdf29006f2fb3062b23b2a94c1be9ad"
server_key_iv: "61608a6db0c58298951c54a2a55eab31"
server_key_version: "encryptionkey@0"
I20260810 20:59:48.002491 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "6a7cd961adb545d1844a518fe9c5a9a4"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "7fdf29006f2fb3062b23b2a94c1be9ad"
server_key_iv: "61608a6db0c58298951c54a2a55eab31"
server_key_version: "encryptionkey@0"
I20260810 20:59:48.003693 18202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:48.004405 18217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:48.004575 18202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:48.004673 18202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "6a7cd961adb545d1844a518fe9c5a9a4"
format_stamp: "Formatted at 2026-08-10 20:59:47 on dist-test-slave-1x4g"
server_key: "7fdf29006f2fb3062b23b2a94c1be9ad"
server_key_iv: "61608a6db0c58298951c54a2a55eab31"
server_key_version: "encryptionkey@0"
I20260810 20:59:48.004768 18202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:48.021461 18202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:48.023159 18202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:48.023307 18202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:48.023518 18202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:48.023854 18202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:48.023907 18202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:48.023957 18202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:48.023985 18202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:48.030308 18202 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:38385
I20260810 20:59:48.030359 18340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:38385 every 8 connection(s)
I20260810 20:59:48.030637 18202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:48.033845 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18202
I20260810 20:59:48.033963 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:48.034175 15136 external_mini_cluster.cc:1428] Setting key 55f5032a4505992c010998836631c387
Aug 10 20:59:48 dist-test-slave-1x4g krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395587, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:48.039495 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:48.031308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:35139 (local address 127.14.200.62:32917)
0810 20:59:48.031374 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:48.031376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:48.031540 (+ 164us) server_negotiation.cc:407] Connection header received
0810 20:59:48.031985 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:48.031986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:48.031994 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:48.032045 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:48.032951 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:48.033421 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:48.034829 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:48.035120 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:48.035175 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:48.037172 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:48.037174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:48.037174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:48.037180 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:48.038397 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:48.038773 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:48.038774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:48.038774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:48.038823 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:48.039026 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:48.039026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:48.039026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:48.039120 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:48.039169 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:48.039374 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 20:59:48.039435 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 20:59:48.039690 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:48.031265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:32917 (local address 127.14.200.3:35139)
0810 20:59:48.031426 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:48.031465 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:48.031898 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:48.032092 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:48.032095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:48.032636 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:48.032867 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:48.032879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:48.033496 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:48.033497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:48.034726 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:48.034730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:48.035196 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:48.035197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:48.035293 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:48.035694 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:48.035702 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:48.037088 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:48.038468 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:48.038475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:48.038476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:48.038695 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:48.038879 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:48.038879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:48.038879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:48.038970 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:48.039172 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:48.039177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:48.039274 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 20:59:48.039350 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 20:59:48.039465 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260810 20:59:48.040187 18341 heartbeater.cc:347] Connected to a master server at 127.14.200.62:32917
I20260810 20:59:48.040295 18341 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:48.040470 18341 heartbeater.cc:511] Master 127.14.200.62:32917 requested a full tablet report, sending...
I20260810 20:59:48.040864 17845 ts_manager.cc:194] Registered new tserver with Master: 6a7cd961adb545d1844a518fe9c5a9a4 (127.14.200.3:38385)
I20260810 20:59:48.041198 17845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:35139
I20260810 20:59:48.045699 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 20:59:48.059531 17886 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 20:59:48.058776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.14.200.62:32917)
0810 20:59:48.058818 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:48.058820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:48.058930 (+ 110us) server_negotiation.cc:407] Connection header received
0810 20:59:48.058978 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:48.058979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:48.058987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:48.059037 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:48.059482 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54000: BlockingRecv error: recv got EOF from 127.0.0.1:54000 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 20:59:48.060484 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17909
I20260810 20:59:48.066865 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18056
I20260810 20:59:48.074086 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18202
I20260810 20:59:48.079841 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 17808
2026-08-10T20:59:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1788 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:48 dist-test-slave-1x4g krb5kdc[18354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:48 dist-test-slave-1x4g krb5kdc[18354](info): set up 2 sockets
Aug 10 20:59:48 dist-test-slave-1x4g krb5kdc[18354](info): commencing operation
krb5kdc: starting...
W20260810 20:59:49.122607 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:49.238969 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33863
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:33863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:49.317862 18367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:49.318084 18367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:49.318120 18367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:49.319269 18367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:49.319332 18367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:49.319356 18367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:49.319386 18367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:49.319416 18367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:49.320896 18367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:33863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:49.321179 18367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:49.321393 18367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:49.325994 18372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:49.326007 18373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:49.326083 18367 server_base.cc:1034] running on GCE node
W20260810 20:59:49.326191 18375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:49.326438 18367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:49.326704 18367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:49.327876 18367 hybrid_clock.cc:648] HybridClock initialized: now 1786395589327861 us; error 48 us; skew 500 ppm
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395589, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:49.330268 18367 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:49.330683 18367 webserver.cc:460] Webserver started at http://127.14.200.62:39527/ using document root <none> and password file <none>
I20260810 20:59:49.330905 18367 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:49.330981 18367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:49.331126 18367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:49.332093 18367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "005072b1d85046089cefcc032bd6a380"
format_stamp: "Formatted at 2026-08-10 20:59:49 on dist-test-slave-1x4g"
server_key: "5f30a2bc587753525b5f0cc7996e87cf"
server_key_iv: "bd5db82284f42d31610d6fa205df6e12"
server_key_version: "encryptionkey@0"
I20260810 20:59:49.332441 18367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "005072b1d85046089cefcc032bd6a380"
format_stamp: "Formatted at 2026-08-10 20:59:49 on dist-test-slave-1x4g"
server_key: "5f30a2bc587753525b5f0cc7996e87cf"
server_key_iv: "bd5db82284f42d31610d6fa205df6e12"
server_key_version: "encryptionkey@0"
I20260810 20:59:49.333817 18367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:49.334525 18385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:49.334690 18367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:49.334798 18367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "005072b1d85046089cefcc032bd6a380"
format_stamp: "Formatted at 2026-08-10 20:59:49 on dist-test-slave-1x4g"
server_key: "5f30a2bc587753525b5f0cc7996e87cf"
server_key_iv: "bd5db82284f42d31610d6fa205df6e12"
server_key_version: "encryptionkey@0"
I20260810 20:59:49.334869 18367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:49.372983 18367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:49.374707 18367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:49.374881 18367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:49.379115 18367 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:33863
I20260810 20:59:49.379154 18439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:33863 every 8 connection(s)
I20260810 20:59:49.379696 18367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:49.380316 18440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:49.382402 18440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: Bootstrap starting.
I20260810 20:59:49.382889 18440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:49.383121 18440 log.cc:824] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:49.383791 18440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: No bootstrap required, opened a new log
I20260810 20:59:49.383859 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18367
I20260810 20:59:49.383970 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:49.384152 15136 external_mini_cluster.cc:1428] Setting key 751a8896725d7978717526edb344ade5
I20260810 20:59:49.386153 18440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } }
I20260810 20:59:49.386291 18440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:49.386391 18440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005072b1d85046089cefcc032bd6a380, State: Initialized, Role: FOLLOWER
I20260810 20:59:49.386533 18440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } }
I20260810 20:59:49.386616 18440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:49.386647 18440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:49.386694 18440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:49.387320 18440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } }
I20260810 20:59:49.387446 18440 leader_election.cc:302] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 005072b1d85046089cefcc032bd6a380; no voters:
I20260810 20:59:49.387616 18440 leader_election.cc:288] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:49.387671 18445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:49.387849 18445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [term 1 LEADER]: Becoming Leader. State: Replica: 005072b1d85046089cefcc032bd6a380, State: Running, Role: LEADER
I20260810 20:59:49.387847 18440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:49.388038 18445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } }
W20260810 20:59:49.389969 18464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 20:59:49.390053 18464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 20:59:49.389453 18447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 005072b1d85046089cefcc032bd6a380. Latest consensus state: current_term: 1 leader_uuid: "005072b1d85046089cefcc032bd6a380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } } }
I20260810 20:59:49.390175 18447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:49.389449 18446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "005072b1d85046089cefcc032bd6a380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005072b1d85046089cefcc032bd6a380" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33863 } } }
I20260810 20:59:49.390408 18466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:49.390439 18446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:49.390599 18466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:49.392042 18466 catalog_manager.cc:1347] Generated new cluster ID: 53b2b94538d342a8881d60ab60f21e6b
I20260810 20:59:49.392092 18466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:49.397298 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:49.384628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.14.200.62:33863)
0810 20:59:49.384844 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:49.384851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:49.384877 (+ 26us) server_negotiation.cc:407] Connection header received
0810 20:59:49.386145 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:49.386151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:49.386197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:49.386336 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:49.387146 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:49.388534 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:49.389643 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:49.390294 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:49.390393 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:49.393210 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:49.393214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:49.393214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:49.393226 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:49.395141 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:49.395532 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:49.395535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:49.395535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:49.395682 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:49.395912 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:49.395913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:49.395913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:49.396206 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:49.396380 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:49.396965 (+ 585us) server_negotiation.cc:299] Negotiation successful
0810 20:59:49.397103 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260810 20:59:49.398111 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 20:59:49.411511 18466 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:49.412137 18466 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:49.414986 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:49.411105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.14.200.62:33863)
0810 20:59:49.411140 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:49.411141 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:49.411259 (+ 118us) server_negotiation.cc:407] Connection header received
0810 20:59:49.411341 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:49.411342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:49.411352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:49.411419 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:49.411800 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:49.412448 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:49.413472 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:49.413784 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:49.413838 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:49.414507 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:49.414508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:49.414509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:49.414707 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:49.414723 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:49.414729 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:49.414786 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:49.414879 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 20:59:49.414925 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 20:59:49.420073 18466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 005072b1d85046089cefcc032bd6a380: Generated new TSK 0
I20260810 20:59:49.420239 18466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 20:59:49.423475 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18367
2026-08-10T20:59:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1342 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18479](info): set up 2 sockets
Aug 10 20:59:49 dist-test-slave-1x4g krb5kdc[18479](info): commencing operation
krb5kdc: starting...
W20260810 20:59:50.472404 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:50 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:50.583158 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35569
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:35569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:50.661741 18493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:50.661973 18493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:50.662009 18493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:50.663076 18493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:50.663134 18493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:50.663157 18493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:50.663184 18493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:50.663211 18493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:50.664736 18493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:35569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:50.665040 18493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:50.665259 18493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:50.669847 18498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:50.669890 18499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:50.669893 18501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:50.669924 18493 server_base.cc:1034] running on GCE node
I20260810 20:59:50.670321 18493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:50.670584 18493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:50.671749 18493 hybrid_clock.cc:648] HybridClock initialized: now 1786395590671740 us; error 46 us; skew 500 ppm
Aug 10 20:59:50 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395590, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:50.674158 18493 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:50.674595 18493 webserver.cc:460] Webserver started at http://127.14.200.62:43573/ using document root <none> and password file <none>
I20260810 20:59:50.674818 18493 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:50.674881 18493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:50.675040 18493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:50.675993 18493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "1aeadbdc4cf64e91addc4a18edea7f63"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ac349672d928db633f44567e0b93d6be"
server_key_iv: "53885d6d33d6cd1663e948f746f03aa8"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.676357 18493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "1aeadbdc4cf64e91addc4a18edea7f63"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ac349672d928db633f44567e0b93d6be"
server_key_iv: "53885d6d33d6cd1663e948f746f03aa8"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.677734 18493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 20:59:50.678417 18510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:50.678586 18493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:50.678687 18493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "1aeadbdc4cf64e91addc4a18edea7f63"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ac349672d928db633f44567e0b93d6be"
server_key_iv: "53885d6d33d6cd1663e948f746f03aa8"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.678756 18493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:50.694420 18493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:50.696074 18493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:50.696225 18493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:50.700336 18493 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:35569
I20260810 20:59:50.700367 18564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:35569 every 8 connection(s)
I20260810 20:59:50.700682 18493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:50.701870 18565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:50.703974 18565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63: Bootstrap starting.
I20260810 20:59:50.704557 18565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:50.704910 18565 log.cc:824] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:50.705614 18565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63: No bootstrap required, opened a new log
I20260810 20:59:50.707232 18565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } }
I20260810 20:59:50.707381 18565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:50.707427 18565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aeadbdc4cf64e91addc4a18edea7f63, State: Initialized, Role: FOLLOWER
I20260810 20:59:50.707595 18565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } }
I20260810 20:59:50.707682 18565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:50.707732 18565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:50.707783 18565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:50.707772 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18493
I20260810 20:59:50.707886 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:50.708062 15136 external_mini_cluster.cc:1428] Setting key 861ebc58f302f149156e7c5421b9fc94
I20260810 20:59:50.708876 18565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } }
I20260810 20:59:50.709010 18565 leader_election.cc:302] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aeadbdc4cf64e91addc4a18edea7f63; no voters:
I20260810 20:59:50.709174 18565 leader_election.cc:288] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:50.709201 18571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:50.709379 18571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [term 1 LEADER]: Becoming Leader. State: Replica: 1aeadbdc4cf64e91addc4a18edea7f63, State: Running, Role: LEADER
I20260810 20:59:50.709511 18571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } }
I20260810 20:59:50.709537 18565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:50.709923 18572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aeadbdc4cf64e91addc4a18edea7f63. Latest consensus state: current_term: 1 leader_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } } }
I20260810 20:59:50.710003 18572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:50.710207 18573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeadbdc4cf64e91addc4a18edea7f63" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35569 } } }
I20260810 20:59:50.710526 18575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:50.710649 18573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:50.711086 18575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:50.712502 18575 catalog_manager.cc:1347] Generated new cluster ID: e7f6f0e293f94275ae425d861bad5994
I20260810 20:59:50.712584 18575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 20:59:50 dist-test-slave-1x4g krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:50.719535 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:50.708527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42892 (local address 127.14.200.62:35569)
0810 20:59:50.708756 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:50.708765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:50.708854 (+ 89us) server_negotiation.cc:407] Connection header received
0810 20:59:50.708963 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:50.708968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:50.709043 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:50.709192 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:50.710066 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.711524 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.712609 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.713119 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.713248 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:50.715721 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:50.715725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:50.715726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:50.715737 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:50.718031 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:50.718398 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:50.718401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:50.718402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:50.718507 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:50.718716 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:50.718717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:50.718717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:50.718943 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:50.719117 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:50.719211 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 20:59:50.719345 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":145,"threads_started":1}
I20260810 20:59:50.738382 18575 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:50.738994 18575 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:50.752573 18575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aeadbdc4cf64e91addc4a18edea7f63: Generated new TSK 0
I20260810 20:59:50.752825 18575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:50.787148 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:50.886540 18593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:50.886739 18593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:50.886775 18593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:50.887861 18593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:50.887923 18593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:50.887964 18593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:50.889390 18593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:50.889672 18593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:50.889886 18593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:50.894819 18601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:50.894835 18598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:50.894912 18599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:50.894969 18593 server_base.cc:1034] running on GCE node
I20260810 20:59:50.895236 18593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:50.895473 18593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:50.896620 18593 hybrid_clock.cc:648] HybridClock initialized: now 1786395590896607 us; error 28 us; skew 500 ppm
Aug 10 20:59:50 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395590, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:50.899205 18593 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:50.899670 18593 webserver.cc:460] Webserver started at http://127.14.200.1:45569/ using document root <none> and password file <none>
I20260810 20:59:50.899885 18593 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:50.899950 18593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:50.900099 18593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:50.901177 18593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "32360769715646939b0747caba65f245"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ba5bd8ea10104ce736ad6e0c1b254123"
server_key_iv: "77ec5ccb15faffce0cc2fbc196d8df4b"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.901526 18593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "32360769715646939b0747caba65f245"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ba5bd8ea10104ce736ad6e0c1b254123"
server_key_iv: "77ec5ccb15faffce0cc2fbc196d8df4b"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.902825 18593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:50.903486 18608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:50.903651 18593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:50.903744 18593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "32360769715646939b0747caba65f245"
format_stamp: "Formatted at 2026-08-10 20:59:50 on dist-test-slave-1x4g"
server_key: "ba5bd8ea10104ce736ad6e0c1b254123"
server_key_iv: "77ec5ccb15faffce0cc2fbc196d8df4b"
server_key_version: "encryptionkey@0"
I20260810 20:59:50.903824 18593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:50.917297 18593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:50.919525 18593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:50.919703 18593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:50.919920 18593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:50.920210 18593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:50.920264 18593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:50.920301 18593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:50.920328 18593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:50.926136 18593 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:34703
I20260810 20:59:50.926187 18732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:34703 every 8 connection(s)
I20260810 20:59:50.926483 18593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 20:59:50 dist-test-slave-1x4g krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395590, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:50.933362 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18593
I20260810 20:59:50.933490 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:50.933729 15136 external_mini_cluster.cc:1428] Setting key 9071f2c03a3a66cd1c874426310f6b09
I20260810 20:59:50.935550 18735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:50.927209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35569 (local address 127.14.200.1:59461)
0810 20:59:50.927363 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:50.927404 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:50.927905 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:50.928193 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:50.928197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:50.928741 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:50.928991 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:50.928996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.929678 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.929678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:50.930402 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:50.930404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.930771 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.930772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:50.930884 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:50.931259 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:50.931266 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:50.932500 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:50.934171 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:50.934179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:50.934181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:50.934475 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:50.934709 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:50.934710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:50.934710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:50.934811 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:50.935081 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:50.935086 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:50.935192 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 20:59:50.935271 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 20:59:50.935396 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260810 20:59:50.935556 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:50.927238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:59461 (local address 127.14.200.62:35569)
0810 20:59:50.927299 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:50.927301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:50.927484 (+ 183us) server_negotiation.cc:407] Connection header received
0810 20:59:50.928057 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:50.928059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:50.928071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:50.928125 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:50.929089 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.929579 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.930496 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:50.930694 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:50.930762 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:50.932577 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:50.932578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:50.932578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:50.932587 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:50.934083 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:50.934568 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:50.934569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:50.934570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:50.934633 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:50.934875 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:50.934876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:50.934876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:50.935016 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:50.935087 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:50.935316 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 20:59:50.935416 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 20:59:50.936331 18733 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35569
I20260810 20:59:50.936460 18733 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:50.936656 18733 heartbeater.cc:511] Master 127.14.200.62:35569 requested a full tablet report, sending...
I20260810 20:59:50.937213 18528 ts_manager.cc:194] Registered new tserver with Master: 32360769715646939b0747caba65f245 (127.14.200.1:34703)
I20260810 20:59:50.937919 18528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:59461
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:50.959745 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:51.036643 18739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:51.036885 18739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:51.036924 18739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:51.038071 18739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:51.038136 18739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:51.038177 18739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:51.039577 18739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:51.039821 18739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:51.040058 18739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:51.044858 18745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:51.044859 18747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:51.044876 18744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:51.045148 18739 server_base.cc:1034] running on GCE node
I20260810 20:59:51.045337 18739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:51.045570 18739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:51.046706 18739 hybrid_clock.cc:648] HybridClock initialized: now 1786395591046695 us; error 25 us; skew 500 ppm
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:51.049147 18739 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:51.049602 18739 webserver.cc:460] Webserver started at http://127.14.200.2:33503/ using document root <none> and password file <none>
I20260810 20:59:51.049831 18739 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:51.049876 18739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:51.050040 18739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:51.051040 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "59135cdd66c84e3687906a49216736bd"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "034e3a1c1cf7b4429a7b2bcd4488c92e"
server_key_iv: "e0abba69e7463e562925dbb50632feaf"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.051391 18739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "59135cdd66c84e3687906a49216736bd"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "034e3a1c1cf7b4429a7b2bcd4488c92e"
server_key_iv: "e0abba69e7463e562925dbb50632feaf"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.052660 18739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 20:59:51.053380 18756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.053556 18739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.053646 18739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "59135cdd66c84e3687906a49216736bd"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "034e3a1c1cf7b4429a7b2bcd4488c92e"
server_key_iv: "e0abba69e7463e562925dbb50632feaf"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.053716 18739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:51.075359 18739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:51.077698 18739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:51.077859 18739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:51.078101 18739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:51.078402 18739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:51.078436 18739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.078459 18739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:51.078505 18739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.084492 18739 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:33229
I20260810 20:59:51.084540 18873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:33229 every 8 connection(s)
I20260810 20:59:51.084897 18739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:51.093688 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:51.085520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:32773 (local address 127.14.200.62:35569)
0810 20:59:51.085566 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:51.085568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:51.085812 (+ 244us) server_negotiation.cc:407] Connection header received
0810 20:59:51.086332 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:51.086334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:51.086347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:51.086399 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:51.087337 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.087852 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.088905 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.089109 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.089193 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:51.091237 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:51.091239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:51.091239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:51.091249 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:51.092475 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.092889 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.092889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.092890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.092943 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.093167 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.093167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.093167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.093262 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:51.093311 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:51.093554 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 20:59:51.093625 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 20:59:51.093825 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:51.085485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35569 (local address 127.14.200.2:32773)
0810 20:59:51.085640 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:51.085707 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:51.086229 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:51.086463 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:51.086467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:51.087045 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:51.087231 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:51.087236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.087961 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.087962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:51.088758 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:51.088760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.089205 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.089206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:51.089361 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:51.089747 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:51.089755 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:51.091147 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:51.092545 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:51.092551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:51.092553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:51.092823 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:51.093021 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:51.093022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:51.093022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:51.093115 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:51.093316 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:51.093320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:51.093441 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 20:59:51.093530 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 20:59:51.093646 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260810 20:59:51.094347 18874 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35569
I20260810 20:59:51.094429 18874 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:51.094586 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18739
I20260810 20:59:51.094630 18874 heartbeater.cc:511] Master 127.14.200.62:35569 requested a full tablet report, sending...
I20260810 20:59:51.094705 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:51.094955 15136 external_mini_cluster.cc:1428] Setting key 2964103636dd9e68b05101e76ea2e304
I20260810 20:59:51.095054 18528 ts_manager.cc:194] Registered new tserver with Master: 59135cdd66c84e3687906a49216736bd (127.14.200.2:33229)
I20260810 20:59:51.095588 18528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:32773
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:51.119477 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:51.240916 18879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:51.241088 18879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:51.241110 18879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:51.242095 18879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:51.242130 18879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:51.242153 18879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:51.243381 18879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:51.243564 18879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:51.243719 18879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:51.248379 18885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:51.248386 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:51.248384 18884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:51.248714 18879 server_base.cc:1034] running on GCE node
I20260810 20:59:51.248919 18879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:51.249135 18879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:51.250268 18879 hybrid_clock.cc:648] HybridClock initialized: now 1786395591250257 us; error 27 us; skew 500 ppm
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:51.252482 18879 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:51.252926 18879 webserver.cc:460] Webserver started at http://127.14.200.3:37331/ using document root <none> and password file <none>
I20260810 20:59:51.253154 18879 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:51.253216 18879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:51.253338 18879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:51.254287 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "a7132b59124c4b9eaeccaec319ecb033"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "8b02465cb973789ecf70a901e9ebcb93"
server_key_iv: "571425a5295c3e170799d547a6d1129a"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.254658 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "a7132b59124c4b9eaeccaec319ecb033"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "8b02465cb973789ecf70a901e9ebcb93"
server_key_iv: "571425a5295c3e170799d547a6d1129a"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.255981 18879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:51.256858 18894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.257028 18879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:51.257123 18879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "a7132b59124c4b9eaeccaec319ecb033"
format_stamp: "Formatted at 2026-08-10 20:59:51 on dist-test-slave-1x4g"
server_key: "8b02465cb973789ecf70a901e9ebcb93"
server_key_iv: "571425a5295c3e170799d547a6d1129a"
server_key_version: "encryptionkey@0"
I20260810 20:59:51.257206 18879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:51.269297 18879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:51.270936 18879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:51.271067 18879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:51.271274 18879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:51.271559 18879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:51.271590 18879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.271610 18879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:51.271649 18879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:51.277338 18879 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:45297
I20260810 20:59:51.277390 19016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:45297 every 8 connection(s)
I20260810 20:59:51.277680 18879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:51.285269 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 18879
I20260810 20:59:51.285364 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:51.285568 15136 external_mini_cluster.cc:1428] Setting key a1286c76935952b4e55a832bc3c1e1b9
I20260810 20:59:51.286332 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:51.278299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:46371 (local address 127.14.200.62:35569)
0810 20:59:51.278359 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:51.278361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:51.278543 (+ 182us) server_negotiation.cc:407] Connection header received
0810 20:59:51.279044 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:51.279045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:51.279055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:51.279102 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:51.279927 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.280368 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.281467 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.281655 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.281741 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:51.283683 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:51.283684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:51.283684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:51.283691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:51.284921 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.285333 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.285334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.285334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.285401 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.285685 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.285686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.285686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.285831 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:51.285902 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:51.286174 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 20:59:51.286252 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 20:59:51.286485 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:51.278264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35569 (local address 127.14.200.3:46371)
0810 20:59:51.278416 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:51.278455 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:51.278958 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:51.279157 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:51.279160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:51.279683 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:51.279860 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:51.279864 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.280440 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.280440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:51.281368 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:51.281370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.281765 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.281766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:51.281898 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:51.282302 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:51.282310 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:51.283610 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:51.285001 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:51.285008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:51.285009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:51.285234 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:51.285490 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:51.285491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:51.285491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:51.285595 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:51.285892 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:51.285899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:51.286049 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 20:59:51.286142 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 20:59:51.286279 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260810 20:59:51.286899 19017 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35569
I20260810 20:59:51.286967 19017 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:51.287089 19017 heartbeater.cc:511] Master 127.14.200.62:35569 requested a full tablet report, sending...
I20260810 20:59:51.287416 18526 ts_manager.cc:194] Registered new tserver with Master: a7132b59124c4b9eaeccaec319ecb033 (127.14.200.3:45297)
I20260810 20:59:51.287783 18526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:46371
I20260810 20:59:51.297397 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395591, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:51.320205 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:51.313772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.14.200.62:35569)
0810 20:59:51.313840 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:51.313843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:51.313982 (+ 139us) server_negotiation.cc:407] Connection header received
0810 20:59:51.314033 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:51.314034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:51.314043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:51.314090 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:51.314631 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.315113 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.315787 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:51.315955 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:51.316002 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:51.318157 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:51.318159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:51.318159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:51.318165 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:51.319286 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.319584 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.319585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.319585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.319631 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:51.319787 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:51.319787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:51.319787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:51.319916 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:51.319987 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:51.320068 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 20:59:51.320122 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 20:59:51.320459 18526 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42908
I20260810 20:59:51.320902 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18593
I20260810 20:59:51.327867 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18739
I20260810 20:59:51.333948 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18879
I20260810 20:59:51.339690 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 18493
2026-08-10T20:59:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1916 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[19029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[19029](info): set up 2 sockets
Aug 10 20:59:51 dist-test-slave-1x4g krb5kdc[19029](info): commencing operation
krb5kdc: starting...
W20260810 20:59:52.385136 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:52.498775 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39321
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:52.575269 19043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:52.575497 19043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:52.575537 19043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:52.576701 19043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:52.576764 19043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:52.576822 19043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:52.576857 19043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:52.576884 19043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:52.578408 19043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:52.578678 19043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:52.578890 19043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:52.583288 19049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.583312 19051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.583312 19048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:52.583585 19043 server_base.cc:1034] running on GCE node
I20260810 20:59:52.583796 19043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:52.584069 19043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:52.585263 19043 hybrid_clock.cc:648] HybridClock initialized: now 1786395592585252 us; error 38 us; skew 500 ppm
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:52.587639 19043 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:52.588069 19043 webserver.cc:460] Webserver started at http://127.14.200.62:39117/ using document root <none> and password file <none>
I20260810 20:59:52.588317 19043 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:52.588382 19043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:52.588527 19043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:52.589583 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "98d62732e77b401d8054dc6082e8484c"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "54a057ec31892dd1c5d5cdad77545cf9"
server_key_iv: "6c2ee40ad962a99715a41c3f948fe7f2"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.589926 19043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "98d62732e77b401d8054dc6082e8484c"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "54a057ec31892dd1c5d5cdad77545cf9"
server_key_iv: "6c2ee40ad962a99715a41c3f948fe7f2"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.591136 19043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:52.591773 19060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.591930 19043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:52.592018 19043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "98d62732e77b401d8054dc6082e8484c"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "54a057ec31892dd1c5d5cdad77545cf9"
server_key_iv: "6c2ee40ad962a99715a41c3f948fe7f2"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.592092 19043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:52.604614 19043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:52.606781 19043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:52.606923 19043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:52.610997 19043 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:39321
I20260810 20:59:52.611035 19117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:39321 every 8 connection(s)
I20260810 20:59:52.611526 19043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:52.612136 19118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:52.613261 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19043
I20260810 20:59:52.613343 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:52.613519 15136 external_mini_cluster.cc:1428] Setting key 7e8a7dc61ba307fbefffe7875d7e76d3
I20260810 20:59:52.614812 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c: Bootstrap starting.
I20260810 20:59:52.615459 19118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:52.615785 19118 log.cc:824] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:52.616462 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c: No bootstrap required, opened a new log
I20260810 20:59:52.618319 19118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } }
I20260810 20:59:52.618476 19118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:52.618525 19118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d62732e77b401d8054dc6082e8484c, State: Initialized, Role: FOLLOWER
I20260810 20:59:52.618659 19118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } }
I20260810 20:59:52.618731 19118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:52.618772 19118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:52.618814 19118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:52.619270 19118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } }
I20260810 20:59:52.619379 19118 leader_election.cc:302] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d62732e77b401d8054dc6082e8484c; no voters:
I20260810 20:59:52.619555 19118 leader_election.cc:288] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:52.619580 19123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:52.619685 19123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [term 1 LEADER]: Becoming Leader. State: Replica: 98d62732e77b401d8054dc6082e8484c, State: Running, Role: LEADER
I20260810 20:59:52.619767 19123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } }
I20260810 20:59:52.619796 19118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:52.620273 19125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d62732e77b401d8054dc6082e8484c. Latest consensus state: current_term: 1 leader_uuid: "98d62732e77b401d8054dc6082e8484c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } } }
I20260810 20:59:52.620529 19125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:52.620443 19124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d62732e77b401d8054dc6082e8484c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d62732e77b401d8054dc6082e8484c" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39321 } } }
I20260810 20:59:52.620831 19124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:52.620998 19132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:52.621476 19132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:52.623157 19132 catalog_manager.cc:1347] Generated new cluster ID: de16dd7b615a47298f30eacddf4b645c
I20260810 20:59:52.623209 19132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:52.624292 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:52.614047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.14.200.62:39321)
0810 20:59:52.614236 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:52.614246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:52.614281 (+ 35us) server_negotiation.cc:407] Connection header received
0810 20:59:52.614812 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:52.614821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:52.614880 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:52.615041 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:52.615623 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.616586 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.617405 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.617832 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.617966 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:52.620447 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:52.620452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:52.620452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:52.620464 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:52.622727 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.623127 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.623129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.623130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.623227 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.623434 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.623435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.623435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.623664 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:52.623798 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:52.623938 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 20:59:52.624095 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260810 20:59:52.640444 19132 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:52.641160 19132 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:52.647565 19132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98d62732e77b401d8054dc6082e8484c: Generated new TSK 0
I20260810 20:59:52.647722 19132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:52.681458 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:52.760289 19146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:52.760545 19146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:52.760586 19146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:52.761839 19146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:52.761897 19146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:52.761945 19146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:52.763334 19146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:52.763584 19146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:52.763798 19146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:52.768581 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.768579 19155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.768548 19151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:52.768839 19146 server_base.cc:1034] running on GCE node
I20260810 20:59:52.769063 19146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:52.769296 19146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:52.770448 19146 hybrid_clock.cc:648] HybridClock initialized: now 1786395592770443 us; error 32 us; skew 500 ppm
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:52.772879 19146 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:52.773284 19146 webserver.cc:460] Webserver started at http://127.14.200.1:38271/ using document root <none> and password file <none>
I20260810 20:59:52.773496 19146 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:52.773563 19146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:52.773695 19146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:52.774684 19146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "bbc1286c4a564eff9d19a22135a5f0c1"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "555a922cfec9742e878511f25bfe1116"
server_key_iv: "d100453d9eb9e14ba42deab286458163"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.775050 19146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "bbc1286c4a564eff9d19a22135a5f0c1"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "555a922cfec9742e878511f25bfe1116"
server_key_iv: "d100453d9eb9e14ba42deab286458163"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.776357 19146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:52.777112 19162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.777272 19146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.777370 19146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "bbc1286c4a564eff9d19a22135a5f0c1"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "555a922cfec9742e878511f25bfe1116"
server_key_iv: "d100453d9eb9e14ba42deab286458163"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.777453 19146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:52.800432 19146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:52.802196 19146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:52.802343 19146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:52.802554 19146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:52.802850 19146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:52.802899 19146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.802935 19146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:52.802961 19146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.808329 19146 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42651
I20260810 20:59:52.808395 19290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42651 every 8 connection(s)
I20260810 20:59:52.808688 19146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:52.816340 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19146
I20260810 20:59:52.816475 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:52.816689 15136 external_mini_cluster.cc:1428] Setting key 7f70b806d4e35e04adaf3bd871d43b3c
I20260810 20:59:52.817438 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:52.809361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:42177 (local address 127.14.200.62:39321)
0810 20:59:52.809423 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:52.809424 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:52.809640 (+ 216us) server_negotiation.cc:407] Connection header received
0810 20:59:52.810511 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:52.810512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:52.810524 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:52.810573 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:52.811132 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.811610 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.812486 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.812691 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.812757 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:52.814742 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:52.814743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:52.814744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:52.814751 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:52.815894 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.816280 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.816281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.816281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.816338 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.816674 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.816675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.816676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.816850 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:52.816944 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:52.817235 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 20:59:52.817341 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 20:59:52.817603 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:52.809339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39321 (local address 127.14.200.1:42177)
0810 20:59:52.809531 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:52.809563 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:52.810384 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:52.810651 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:52.810654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:52.810880 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:52.811041 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:52.811045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.811700 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.811701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:52.812398 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:52.812400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.812796 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.812798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:52.812904 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:52.813359 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:52.813367 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:52.814645 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:52.816005 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:52.816012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:52.816015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:52.816208 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:52.816399 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:52.816399 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:52.816400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:52.816569 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:52.816929 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:52.816936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:52.817099 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 20:59:52.817205 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 20:59:52.817361 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260810 20:59:52.818058 19291 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39321
I20260810 20:59:52.818156 19291 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:52.818313 19291 heartbeater.cc:511] Master 127.14.200.62:39321 requested a full tablet report, sending...
I20260810 20:59:52.818816 19078 ts_manager.cc:194] Registered new tserver with Master: bbc1286c4a564eff9d19a22135a5f0c1 (127.14.200.1:42651)
I20260810 20:59:52.819692 19078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:42177
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:52.845575 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:52.922863 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:52.923043 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:52.923065 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:52.924160 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:52.924194 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:52.924219 19298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:52.925623 19298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:52.925776 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:52.925948 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:52.930847 19304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.930876 19306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:52.930837 19303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:52.930895 19298 server_base.cc:1034] running on GCE node
I20260810 20:59:52.931288 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:52.931555 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:52.932705 19298 hybrid_clock.cc:648] HybridClock initialized: now 1786395592932686 us; error 31 us; skew 500 ppm
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:52.935076 19298 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:52.935531 19298 webserver.cc:460] Webserver started at http://127.14.200.2:43953/ using document root <none> and password file <none>
I20260810 20:59:52.935774 19298 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:52.935842 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:52.935982 19298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:52.937060 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "009efe07933443c188535ab36777447e"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "9c3d4cf05d8053ea6dbacdf71931b001"
server_key_iv: "ba3d1c40bf556d4ce7bb15a88b4c42e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.937417 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "009efe07933443c188535ab36777447e"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "9c3d4cf05d8053ea6dbacdf71931b001"
server_key_iv: "ba3d1c40bf556d4ce7bb15a88b4c42e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.938720 19298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:52.939442 19313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.939606 19298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:52.939687 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "009efe07933443c188535ab36777447e"
format_stamp: "Formatted at 2026-08-10 20:59:52 on dist-test-slave-1x4g"
server_key: "9c3d4cf05d8053ea6dbacdf71931b001"
server_key_iv: "ba3d1c40bf556d4ce7bb15a88b4c42e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:52.939736 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:52.979051 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:52.981400 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:52.981516 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:52.981729 19298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:52.982050 19298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:52.982084 19298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.982126 19298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:52.982151 19298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:52.987773 19298 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:42437
I20260810 20:59:52.987818 19438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:42437 every 8 connection(s)
I20260810 20:59:52.988139 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 20:59:52.990061 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19298
I20260810 20:59:52.990164 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:52.990347 15136 external_mini_cluster.cc:1428] Setting key b61766da77aa79c04790e7dd331b9a2b
Aug 10 20:59:52 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395592, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:52.998224 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:52.988877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:45979 (local address 127.14.200.62:39321)
0810 20:59:52.988934 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:52.988936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:52.989142 (+ 206us) server_negotiation.cc:407] Connection header received
0810 20:59:52.990072 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:52.990073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:52.990086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:52.990150 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:52.990795 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.991286 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.992240 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.992543 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.992616 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:52.995145 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:52.995148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:52.995149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:52.995159 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:52.996667 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.997244 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.997245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.997245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.997298 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:52.997594 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:52.997595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:52.997595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:52.997695 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:52.997742 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:52.998105 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 20:59:52.998162 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 20:59:52.998481 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:52.988863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39321 (local address 127.14.200.2:45979)
0810 20:59:52.989028 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:52.989064 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:52.989956 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:52.990219 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:52.990223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:52.990469 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:52.990705 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:52.990710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.991382 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.991383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:52.992154 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:52.992156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:52.992645 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:52.992646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:52.992757 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:52.993297 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:52.993306 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:52.995028 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:52.996756 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:52.996764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:52.996766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:52.997165 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:52.997369 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:52.997370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:52.997370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:52.997521 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:52.997760 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:52.997768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:52.997954 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 20:59:52.998072 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 20:59:52.998246 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260810 20:59:52.999014 19439 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39321
I20260810 20:59:52.999107 19439 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:52.999303 19439 heartbeater.cc:511] Master 127.14.200.62:39321 requested a full tablet report, sending...
I20260810 20:59:52.999728 19078 ts_manager.cc:194] Registered new tserver with Master: 009efe07933443c188535ab36777447e (127.14.200.2:42437)
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
I20260810 20:59:53.000097 19078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:45979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:53.019228 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:53.097177 19444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:53.097381 19444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:53.097420 19444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:53.098534 19444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:53.098596 19444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:53.098639 19444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:53.100076 19444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:53.100319 19444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:53.100538 19444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:53.105332 19451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:53.105290 19453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:53.105301 19450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:53.105468 19444 server_base.cc:1034] running on GCE node
I20260810 20:59:53.105695 19444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:53.105922 19444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:53.107067 19444 hybrid_clock.cc:648] HybridClock initialized: now 1786395593107043 us; error 38 us; skew 500 ppm
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395593, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:53.109670 19444 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:53.110136 19444 webserver.cc:460] Webserver started at http://127.14.200.3:42537/ using document root <none> and password file <none>
I20260810 20:59:53.110373 19444 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:53.110440 19444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:53.110601 19444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:53.111560 19444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "8059efeb941b4c3aa660718de2fadeeb"
format_stamp: "Formatted at 2026-08-10 20:59:53 on dist-test-slave-1x4g"
server_key: "4fdcce9df3348aad14aafcc2378bd7cd"
server_key_iv: "b17f428e0b574b39ce132cb5a4ffd731"
server_key_version: "encryptionkey@0"
I20260810 20:59:53.111902 19444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "8059efeb941b4c3aa660718de2fadeeb"
format_stamp: "Formatted at 2026-08-10 20:59:53 on dist-test-slave-1x4g"
server_key: "4fdcce9df3348aad14aafcc2378bd7cd"
server_key_iv: "b17f428e0b574b39ce132cb5a4ffd731"
server_key_version: "encryptionkey@0"
I20260810 20:59:53.113243 19444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:53.113967 19462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:53.114166 19444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:53.114264 19444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "8059efeb941b4c3aa660718de2fadeeb"
format_stamp: "Formatted at 2026-08-10 20:59:53 on dist-test-slave-1x4g"
server_key: "4fdcce9df3348aad14aafcc2378bd7cd"
server_key_iv: "b17f428e0b574b39ce132cb5a4ffd731"
server_key_version: "encryptionkey@0"
I20260810 20:59:53.114341 19444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:53.122491 19444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:53.124269 19444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:53.124424 19444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:53.124644 19444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:53.124953 19444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:53.124987 19444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:53.125007 19444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:53.125020 19444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:53.130800 19444 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:32785
I20260810 20:59:53.130834 19586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:32785 every 8 connection(s)
I20260810 20:59:53.131147 19444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:53.134027 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19444
I20260810 20:59:53.134140 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:53.134310 15136 external_mini_cluster.cc:1428] Setting key 65f6e4b7d91ea0873e80d6e81da1fde7
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395593, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:53.140267 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:53.131848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:50397 (local address 127.14.200.62:39321)
0810 20:59:53.131904 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:53.131906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:53.132133 (+ 227us) server_negotiation.cc:407] Connection header received
0810 20:59:53.133242 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:53.133243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:53.133254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:53.133313 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:53.133873 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.134672 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.135778 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.135969 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.136019 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:53.138115 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:53.138116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:53.138116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:53.138121 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:53.139265 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.139594 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.139595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.139595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.139641 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.139829 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.139830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.139830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.139915 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:53.139961 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:53.140158 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 20:59:53.140212 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 20:59:53.140401 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:53.131813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39321 (local address 127.14.200.3:50397)
0810 20:59:53.131997 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:53.132055 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:53.133154 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:53.133366 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:53.133369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:53.133604 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:53.133792 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:53.133795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.134765 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.134766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:53.135694 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:53.135697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.136039 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.136040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:53.136134 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:53.136605 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:53.136613 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:53.138044 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:53.139331 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:53.139336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:53.139338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:53.139535 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:53.139694 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:53.139694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:53.139694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:53.139781 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:53.139965 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:53.139968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:53.140062 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 20:59:53.140133 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 20:59:53.140233 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260810 20:59:53.140841 19587 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39321
I20260810 20:59:53.140919 19587 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:53.141077 19587 heartbeater.cc:511] Master 127.14.200.62:39321 requested a full tablet report, sending...
I20260810 20:59:53.141379 19078 ts_manager.cc:194] Registered new tserver with Master: 8059efeb941b4c3aa660718de2fadeeb (127.14.200.3:32785)
I20260810 20:59:53.141712 19078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:50397
I20260810 20:59:53.145850 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 20:59:53.169718 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:53.162495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.14.200.1:42651)
0810 20:59:53.162643 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:53.162648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:53.162660 (+ 12us) server_negotiation.cc:407] Connection header received
0810 20:59:53.162693 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:53.162695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:53.162709 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:53.162772 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:53.163290 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.163977 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.164733 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.165178 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.165237 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:53.167416 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:53.167424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:53.167424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:53.167430 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:53.168762 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.169101 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.169104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.169104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.169172 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.169315 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.169316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.169316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.169425 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:53.169517 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:53.169587 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 20:59:53.169645 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:53.176745 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:53.170985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.14.200.62:39321)
0810 20:59:53.171056 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:53.171058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:53.171066 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:53.171113 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:53.171114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:53.171122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:53.171159 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:53.171541 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.171955 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.172505 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:53.172658 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:53.172702 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:53.174663 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:53.174664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:53.174664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:53.174671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:53.175800 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.176096 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.176097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.176097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.176146 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:53.176303 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:53.176303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:53.176303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:53.176447 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:53.176519 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:53.176612 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 20:59:53.176688 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 20:59:53.177000 19078 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34016
I20260810 20:59:53.177338 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19146
I20260810 20:59:53.185683 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19298
I20260810 20:59:53.191334 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19444
I20260810 20:59:53.198261 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19043
2026-08-10T20:59:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1859 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19600](info): set up 2 sockets
Aug 10 20:59:53 dist-test-slave-1x4g krb5kdc[19600](info): commencing operation
krb5kdc: starting...
W20260810 20:59:54.254814 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260810 20:59:54.497591 15136 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.243s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260810 20:59:54.724179 15136 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.226s user 0.001s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260810 20:59:54.944199 15136 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.220s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/test-user.keytab.
W20260810 20:59:55.504627 15136 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.560s user 0.000s sys 0.002s
Aug 10 20:59:55 dist-test-slave-1x4g krb5kdc[19600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
W20260810 20:59:55.905738 15136 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.14.200.62: real 0.378s user 0.000s sys 0.003s
I20260810 20:59:55.906231 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45947
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:55.986620 19615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:55.986805 19615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:55.986827 19615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:55.987986 19615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:55.988026 19615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:55.988040 19615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:55.988077 19615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:55.988106 19615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:55.989574 19615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:55.989805 19615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:55.990013 19615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:55.994462 19620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:55.994479 19622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:55.994521 19624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:55.994678 19615 server_base.cc:1034] running on GCE node
I20260810 20:59:55.994920 19615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:55.995223 19615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:55.996381 19615 hybrid_clock.cc:648] HybridClock initialized: now 1786395595996368 us; error 25 us; skew 500 ppm
Aug 10 20:59:55 dist-test-slave-1x4g krb5kdc[19600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395595, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:55.998970 19615 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:55.999274 19615 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:55.999318 19615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:55.999471 19615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:56.000516 19615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "fdff3a0cb005493a982947898773665a"
format_stamp: "Formatted at 2026-08-10 20:59:55 on dist-test-slave-1x4g"
server_key: "2eca33fc8f2329e8ee9ffa84ad2ca0c2"
server_key_iv: "8843ad6339c7c2b3c2625522793e94c6"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.000958 19615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "fdff3a0cb005493a982947898773665a"
format_stamp: "Formatted at 2026-08-10 20:59:55 on dist-test-slave-1x4g"
server_key: "2eca33fc8f2329e8ee9ffa84ad2ca0c2"
server_key_iv: "8843ad6339c7c2b3c2625522793e94c6"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.002239 19615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 20:59:56.002897 19630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.003065 19615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:56.003165 19615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "fdff3a0cb005493a982947898773665a"
format_stamp: "Formatted at 2026-08-10 20:59:55 on dist-test-slave-1x4g"
server_key: "2eca33fc8f2329e8ee9ffa84ad2ca0c2"
server_key_iv: "8843ad6339c7c2b3c2625522793e94c6"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.003235 19615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:56.027660 19615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:56.029979 19615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:56.030135 19615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:56.034199 19615 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45947
I20260810 20:59:56.034250 19691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45947 every 8 connection(s)
I20260810 20:59:56.034556 19615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:56.035633 19692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:56.038018 19692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: Bootstrap starting.
I20260810 20:59:56.038555 19692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:56.038796 19692 log.cc:824] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:56.039384 19692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: No bootstrap required, opened a new log
I20260810 20:59:56.040879 19692 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } }
I20260810 20:59:56.041020 19692 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:56.041042 19692 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdff3a0cb005493a982947898773665a, State: Initialized, Role: FOLLOWER
I20260810 20:59:56.041165 19692 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } }
I20260810 20:59:56.041242 19692 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:56.041263 19692 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:56.041290 19692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:56.042047 19692 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } }
I20260810 20:59:56.042158 19692 leader_election.cc:302] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdff3a0cb005493a982947898773665a; no voters:
I20260810 20:59:56.042186 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19615
I20260810 20:59:56.042279 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:56.042353 19692 leader_election.cc:288] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:56.042434 19695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:56.042503 15136 external_mini_cluster.cc:1428] Setting key 04e019d6a50903c2c4b5d0ae87068ae8
I20260810 20:59:56.042593 19692 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:56.042599 19695 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [term 1 LEADER]: Becoming Leader. State: Replica: fdff3a0cb005493a982947898773665a, State: Running, Role: LEADER
I20260810 20:59:56.042712 19695 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } }
I20260810 20:59:56.043066 19696 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdff3a0cb005493a982947898773665a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } } }
I20260810 20:59:56.043149 19696 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:56.043172 19698 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdff3a0cb005493a982947898773665a. Latest consensus state: current_term: 1 leader_uuid: "fdff3a0cb005493a982947898773665a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff3a0cb005493a982947898773665a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45947 } } }
I20260810 20:59:56.043339 19698 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:56.043361 19708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:56.043852 19708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:56.046020 19708 catalog_manager.cc:1347] Generated new cluster ID: ddc67cf2f5614ba78dd1b7376725d201
I20260810 20:59:56.046072 19708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:56.053139 19708 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:56.053784 19708 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:56.054878 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.043037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.14.200.62:45947)
0810 20:59:56.043462 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.043471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.043743 (+ 272us) server_negotiation.cc:407] Connection header received
0810 20:59:56.043962 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.043969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.044023 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.044131 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.045157 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.046539 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.047561 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.048172 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.048287 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.051075 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.051079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.051079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:56.051091 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.053145 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.053558 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.053561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.053561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.053681 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.053918 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.053919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.053919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.054197 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.054374 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.054480 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.054645 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":59,"threads_started":1}
I20260810 20:59:56.056087 19708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: Generated new TSK 0
I20260810 20:59:56.056239 19708 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 20:59:56.087292 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:56.170810 19722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:56.171039 19722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:56.171082 19722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:56.172384 19722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:56.172451 19722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:56.172497 19722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:56.174084 19722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:56.174355 19722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:56.174597 19722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:56.179589 19727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.179601 19731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.179586 19729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:56.179937 19722 server_base.cc:1034] running on GCE node
I20260810 20:59:56.180154 19722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:56.180408 19722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:56.181566 19722 hybrid_clock.cc:648] HybridClock initialized: now 1786395596181541 us; error 39 us; skew 500 ppm
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:56.184104 19722 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 20:59:56.184455 19722 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:56.184540 19722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:56.184693 19722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:56.185848 19722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "d214d32265d440919892bbb18658a1fc"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "1dbc7dbaf926c23332fcdb1413755073"
server_key_iv: "3d26e55f6952cf7ee153673d4fd84f09"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.186249 19722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "d214d32265d440919892bbb18658a1fc"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "1dbc7dbaf926c23332fcdb1413755073"
server_key_iv: "3d26e55f6952cf7ee153673d4fd84f09"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.187707 19722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:56.188380 19738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.188548 19722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:56.188647 19722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "d214d32265d440919892bbb18658a1fc"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "1dbc7dbaf926c23332fcdb1413755073"
server_key_iv: "3d26e55f6952cf7ee153673d4fd84f09"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.188724 19722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:56.205610 19722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:56.207695 19722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:56.207857 19722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:56.208091 19722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:56.208432 19722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:56.208488 19722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.208525 19722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:56.208552 19722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.214377 19722 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:34459
I20260810 20:59:56.214488 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:34459 every 8 connection(s)
I20260810 20:59:56.214740 19722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:56.222571 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19722
I20260810 20:59:56.222702 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:56.222925 15136 external_mini_cluster.cc:1428] Setting key 37965790d30ce81918d6f13e395f7a59
I20260810 20:59:56.225170 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.215533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:42431 (local address 127.14.200.62:45947)
0810 20:59:56.215630 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.215632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.215843 (+ 211us) server_negotiation.cc:407] Connection header received
0810 20:59:56.216397 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.216398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.216408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.216460 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.217678 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.218170 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.219181 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.219369 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.219419 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.221926 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.221928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.221928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:56.221936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.223562 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.224025 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.224026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.224026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.224101 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.224394 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.224395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.224395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.224523 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.224608 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.224975 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.225073 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 20:59:56.225409 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.215532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45947 (local address 127.14.200.1:42431)
0810 20:59:56.215726 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.215769 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.216297 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.216517 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.216520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.216894 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:56.217583 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.217590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.218273 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.218274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.219097 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.219100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.219445 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.219446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.219551 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.220094 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:56.220103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:56.221827 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:56.223678 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.223685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.223687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.223920 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.224194 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.224194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.224195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.224318 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.224598 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:56.224606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:56.224770 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.224949 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.225120 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260810 20:59:56.226035 19867 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45947
I20260810 20:59:56.226156 19867 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:56.226341 19867 heartbeater.cc:511] Master 127.14.200.62:45947 requested a full tablet report, sending...
I20260810 20:59:56.226935 19648 ts_manager.cc:194] Registered new tserver with Master: d214d32265d440919892bbb18658a1fc (127.14.200.1:34459)
I20260810 20:59:56.227568 19648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:42431
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 20:59:56.251868 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:56.334165 19872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:56.334405 19872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:56.334452 19872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:56.335613 19872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:56.335677 19872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:56.335723 19872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:56.337272 19872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:56.337543 19872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:56.337833 19872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:56.343195 19880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.343195 19877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.343338 19878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:56.343400 19872 server_base.cc:1034] running on GCE node
I20260810 20:59:56.343727 19872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:56.343978 19872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:56.345157 19872 hybrid_clock.cc:648] HybridClock initialized: now 1786395596345147 us; error 28 us; skew 500 ppm
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:56.347713 19872 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 20:59:56.348068 19872 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:56.348116 19872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:56.348205 19872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:56.349241 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "fe51eff673b24b2b8191a28d4d617fed"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "f8e6958e2195ae9333650693111b0c1a"
server_key_iv: "47a5cee70d45c7aa8fe683eb74d9829a"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.349607 19872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "fe51eff673b24b2b8191a28d4d617fed"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "f8e6958e2195ae9333650693111b0c1a"
server_key_iv: "47a5cee70d45c7aa8fe683eb74d9829a"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.351107 19872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:56.351775 19887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.351954 19872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:56.352090 19872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "fe51eff673b24b2b8191a28d4d617fed"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "f8e6958e2195ae9333650693111b0c1a"
server_key_iv: "47a5cee70d45c7aa8fe683eb74d9829a"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.352164 19872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:56.369143 19872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:56.370967 19872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:56.371127 19872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:56.371371 19872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:56.371691 19872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:56.371744 19872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.371788 19872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:56.371825 19872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.377892 19872 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46087
I20260810 20:59:56.377933 20011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46087 every 8 connection(s)
I20260810 20:59:56.378264 19872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:56.386682 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.378928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:52345 (local address 127.14.200.62:45947)
0810 20:59:56.379008 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.379010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.379212 (+ 202us) server_negotiation.cc:407] Connection header received
0810 20:59:56.379712 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.379714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.379723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.379773 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.380652 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.381195 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.382109 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.382283 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.382331 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.384320 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.384321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.384322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:56.384328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.385607 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.385949 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.385950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.385951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.386002 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.386204 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.386204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.386204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.386299 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.386347 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.386559 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.386620 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 20:59:56.386871 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.378891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45947 (local address 127.14.200.2:52345)
0810 20:59:56.379063 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.379100 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.379627 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.379827 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.379831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.380090 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:56.380579 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.380584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.381270 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.381271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.382041 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.382043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.382352 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.382353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.382447 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.382855 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:56.382864 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:56.384234 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:56.385674 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.385680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.385682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.385886 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.386057 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.386058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.386058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.386154 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.386350 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:56.386354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:56.386458 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.386534 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.386644 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260810 20:59:56.387415 20012 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45947
I20260810 20:59:56.387524 20012 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:56.387681 20012 heartbeater.cc:511] Master 127.14.200.62:45947 requested a full tablet report, sending...
I20260810 20:59:56.387701 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 19872
I20260810 20:59:56.387814 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:56.388034 15136 external_mini_cluster.cc:1428] Setting key d2ccbfa40bbf84b9194f2cb93b312630
I20260810 20:59:56.388098 19648 ts_manager.cc:194] Registered new tserver with Master: fe51eff673b24b2b8191a28d4d617fed (127.14.200.2:46087)
I20260810 20:59:56.388546 19648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:52345
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 20:59:56.414031 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:56.493239 20017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:56.493453 20017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:56.493490 20017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:56.494601 20017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:56.494668 20017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:56.494714 20017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:56.496165 20017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:56.496398 20017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:56.496654 20017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:56.501269 20028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.501278 20024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.501278 20025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:56.501515 20017 server_base.cc:1034] running on GCE node
I20260810 20:59:56.501736 20017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:56.501927 20017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:56.503058 20017 hybrid_clock.cc:648] HybridClock initialized: now 1786395596503051 us; error 30 us; skew 500 ppm
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:56.508473 20017 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 20:59:56.508883 20017 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:56.508971 20017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:56.509112 20017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:56.510179 20017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "b78dfcb2b7a248caaf2c5d05cfab8328"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "6e6a0f250399a3e48124eb9dfa7ab166"
server_key_iv: "66193d1c91fbf93ff3542dcf12be1cda"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.510569 20017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "b78dfcb2b7a248caaf2c5d05cfab8328"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "6e6a0f250399a3e48124eb9dfa7ab166"
server_key_iv: "66193d1c91fbf93ff3542dcf12be1cda"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.511946 20017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 20:59:56.512661 20034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.512848 20017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:56.512940 20017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "b78dfcb2b7a248caaf2c5d05cfab8328"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "6e6a0f250399a3e48124eb9dfa7ab166"
server_key_iv: "66193d1c91fbf93ff3542dcf12be1cda"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.513015 20017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:56.524281 20017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:56.526188 20017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:56.526329 20017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:56.526531 20017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:56.526813 20017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:56.526859 20017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.526896 20017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:56.526921 20017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.532698 20017 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:41239
I20260810 20:59:56.532754 20161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:41239 every 8 connection(s)
I20260810 20:59:56.533003 20017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
Aug 10 20:59:56 dist-test-slave-1x4g krb5kdc[19600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395596, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 20:59:56.538594 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20017
I20260810 20:59:56.538671 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:56.538897 15136 external_mini_cluster.cc:1428] Setting key 4440250f29b389ceab0ec1b7d0509b4c
I20260810 20:59:56.541522 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.533635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:41247 (local address 127.14.200.62:45947)
0810 20:59:56.533678 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.533680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.533905 (+ 225us) server_negotiation.cc:407] Connection header received
0810 20:59:56.534335 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.534335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.534345 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.534407 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.535239 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.535730 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.536604 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.536817 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.536863 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.538729 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.538731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.538731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:56.538740 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.540356 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.540703 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.540703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.540704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.540756 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.541032 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.541033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.541033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.541132 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.541181 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.541395 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.541464 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 20:59:56.541654 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.533646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45947 (local address 127.14.200.3:41247)
0810 20:59:56.533782 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.533812 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.534225 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.534462 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.534466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.534704 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:56.535166 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.535171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.535807 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.535808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.536530 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.536532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.536883 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.536884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.536986 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.537358 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 20:59:56.537365 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:56.538620 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:56.540427 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.540433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.540434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.540639 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.540870 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 20:59:56.540870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 20:59:56.540871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 20:59:56.540976 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 20:59:56.541188 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:56.541192 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:56.541292 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.541366 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.541482 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260810 20:59:56.542124 20162 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45947
I20260810 20:59:56.542235 20162 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:56.542368 20162 heartbeater.cc:511] Master 127.14.200.62:45947 requested a full tablet report, sending...
I20260810 20:59:56.542812 19648 ts_manager.cc:194] Registered new tserver with Master: b78dfcb2b7a248caaf2c5d05cfab8328 (127.14.200.3:41239)
I20260810 20:59:56.543184 19648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:41247
I20260810 20:59:56.550855 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 20:59:56.558135 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.552173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.14.200.62:45947)
0810 20:59:56.552248 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.552250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.552292 (+ 42us) server_negotiation.cc:407] Connection header received
0810 20:59:56.552343 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.552344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.552351 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.552395 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.552971 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.553392 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.554020 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.554252 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.554324 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.555427 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.555428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.555428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 20:59:56.555439 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.557034 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.557323 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.557324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.557324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.557372 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 20:59:56.557516 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 20:59:56.557517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 20:59:56.557517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 20:59:56.557629 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.557697 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.557988 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.558046 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 20:59:56.559580 19648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:56.561116 19648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:56.567345 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:41239 (local address 127.0.0.1:57598)
0810 20:59:56.563388 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.563420 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.563482 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.563872 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.563874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.563877 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:56.563938 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.563940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.565292 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.565293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.566361 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.566363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.567025 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.567026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.567116 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.567130 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.567232 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.567284 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":5,"thread_start_us":48,"threads_started":1}
I20260810 20:59:56.567355 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.14.200.3:41239)
0810 20:59:56.563730 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.563733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.563740 (+ 7us) server_negotiation.cc:407] Connection header received
0810 20:59:56.563765 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.563766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.563777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.563835 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:56.564007 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.565185 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.566446 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.566923 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.567011 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.567068 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.567249 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.567284 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260810 20:59:56.567515 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46087 (local address 127.0.0.1:46276)
0810 20:59:56.564016 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.564027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.564148 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.564451 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.564452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.564454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:56.564483 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.564484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.565520 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.565521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.566356 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.566359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.567293 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.567294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.567347 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.567351 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.567401 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.567441 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":69,"threads_started":1}
I20260810 20:59:56.567773 20180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.14.200.2:46087)
0810 20:59:56.563644 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.563648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.564078 (+ 430us) server_negotiation.cc:407] Connection header received
0810 20:59:56.564267 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.564270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.564287 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.564341 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:56.564556 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.565397 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.566454 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.567154 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.567217 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.567261 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.567656 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.567692 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260810 20:59:56.568018 20088 tablet_service.cc:1467] Processing CreateTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 (DEFAULT_TABLE table=test-table [id=4ad22c70c3914e8fb46677af8504ad8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:56.568337 20088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c49ec84a0b59462eb0dbda67e3163c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:56.568312 19938 tablet_service.cc:1467] Processing CreateTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 (DEFAULT_TABLE table=test-table [id=4ad22c70c3914e8fb46677af8504ad8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:56.568645 19938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c49ec84a0b59462eb0dbda67e3163c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:56.568881 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34459 (local address 127.0.0.1:40376)
0810 20:59:56.563484 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.563494 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.563541 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.563924 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.563925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.563927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:56.563998 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.564000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.566454 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.566455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.567717 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.567720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.568654 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.568655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.568714 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.568717 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.568791 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.568823 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260810 20:59:56.568959 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.563424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.14.200.1:34459)
0810 20:59:56.563755 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.563758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.563766 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:56.563797 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.563799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.563810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.563920 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:56.564089 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.565185 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.567779 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.568515 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.568572 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.568620 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.568838 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.568884 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":70,"threads_started":1}
I20260810 20:59:56.570997 20182 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Bootstrap starting.
I20260810 20:59:56.571025 20183 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Bootstrap starting.
I20260810 20:59:56.571659 20182 tablet_bootstrap.cc:654] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:56.571727 19790 tablet_service.cc:1467] Processing CreateTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 (DEFAULT_TABLE table=test-table [id=4ad22c70c3914e8fb46677af8504ad8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:56.571884 20183 tablet_bootstrap.cc:654] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:56.572089 19790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c49ec84a0b59462eb0dbda67e3163c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:56.572268 20182 log.cc:824] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:56.572458 20183 log.cc:824] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:56.573130 20183 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: No bootstrap required, opened a new log
I20260810 20:59:56.573154 20182 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: No bootstrap required, opened a new log
I20260810 20:59:56.573215 20182 ts_tablet_manager.cc:1397] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:56.573216 20183 ts_tablet_manager.cc:1397] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:56.574568 20183 raft_consensus.cc:348] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.574743 20183 raft_consensus.cc:374] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:56.574788 20183 raft_consensus.cc:729] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe51eff673b24b2b8191a28d4d617fed, State: Initialized, Role: FOLLOWER
I20260810 20:59:56.574896 20183 consensus_queue.cc:260] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.575186 20183 ts_tablet_manager.cc:1428] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 20:59:56.575261 20012 heartbeater.cc:503] Master 127.14.200.62:45947 was elected leader, sending a full tablet report...
I20260810 20:59:56.575232 20182 raft_consensus.cc:348] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.575428 20182 raft_consensus.cc:374] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:56.575486 20182 raft_consensus.cc:729] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b78dfcb2b7a248caaf2c5d05cfab8328, State: Initialized, Role: FOLLOWER
I20260810 20:59:56.575593 20182 consensus_queue.cc:260] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.575744 20187 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Bootstrap starting.
I20260810 20:59:56.575841 20182 ts_tablet_manager.cc:1428] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 20:59:56.576025 20162 heartbeater.cc:503] Master 127.14.200.62:45947 was elected leader, sending a full tablet report...
I20260810 20:59:56.576489 20187 tablet_bootstrap.cc:654] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:56.577077 20187 log.cc:824] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:56.577685 20187 tablet_bootstrap.cc:492] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: No bootstrap required, opened a new log
I20260810 20:59:56.577754 20187 ts_tablet_manager.cc:1397] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 20:59:56.579093 20187 raft_consensus.cc:348] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.579272 20187 raft_consensus.cc:374] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:56.579304 20187 raft_consensus.cc:729] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d214d32265d440919892bbb18658a1fc, State: Initialized, Role: FOLLOWER
I20260810 20:59:56.579391 20187 consensus_queue.cc:260] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.579591 20187 ts_tablet_manager.cc:1428] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 20:59:56.579646 19867 heartbeater.cc:503] Master 127.14.200.62:45947 was elected leader, sending a full tablet report...
W20260810 20:59:56.628824 20013 tablet.cc:2369] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 20:59:56.715530 19868 tablet.cc:2369] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 20:59:56.727558 20186 raft_consensus.cc:482] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:56.727674 20186 raft_consensus.cc:504] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.727998 20186 leader_election.cc:288] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b78dfcb2b7a248caaf2c5d05cfab8328 (127.14.200.3:41239), d214d32265d440919892bbb18658a1fc (127.14.200.1:34459)
I20260810 20:59:56.730738 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.728213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34459 (local address 127.14.200.2:54049)
0810 20:59:56.728281 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.728294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.728344 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.728525 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.728526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.728528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:56.728585 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.728587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.729227 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.729228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.729954 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.729956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.730479 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.730555 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.730557 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.730622 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.730657 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":1}
I20260810 20:59:56.730738 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.728299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54049 (local address 127.14.200.1:34459)
0810 20:59:56.728343 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.728346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.728355 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:56.728442 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.728443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.728450 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.728513 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:56.728651 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.729143 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730054 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.730398 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730479 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.730488 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.730629 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.730678 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 20:59:56.730916 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.728288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39113 (local address 127.14.200.3:41239)
0810 20:59:56.728352 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.728355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.728510 (+ 155us) server_negotiation.cc:407] Connection header received
0810 20:59:56.728589 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.728590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.728605 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.728665 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 20:59:56.728857 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.729309 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730206 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.730576 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730658 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.730677 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.730818 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.730853 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 20:59:56.730917 20192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.728216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:41239 (local address 127.14.200.2:39113)
0810 20:59:56.728425 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.728435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:56.728521 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:56.728667 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:56.728668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:56.728670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 20:59:56.728751 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.728753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.729397 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.729398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.730116 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:56.730118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.730658 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.730659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:56.730737 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.730740 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:56.730810 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 20:59:56.730842 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":12,"thread_start_us":108,"threads_started":1}
I20260810 20:59:56.731091 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c49ec84a0b59462eb0dbda67e3163c66" candidate_uuid: "fe51eff673b24b2b8191a28d4d617fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d214d32265d440919892bbb18658a1fc" is_pre_election: true
I20260810 20:59:56.731236 19813 raft_consensus.cc:2454] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe51eff673b24b2b8191a28d4d617fed in term 0.
I20260810 20:59:56.731257 20110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c49ec84a0b59462eb0dbda67e3163c66" candidate_uuid: "fe51eff673b24b2b8191a28d4d617fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" is_pre_election: true
I20260810 20:59:56.731436 20110 raft_consensus.cc:2454] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe51eff673b24b2b8191a28d4d617fed in term 0.
I20260810 20:59:56.731482 19891 leader_election.cc:302] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d214d32265d440919892bbb18658a1fc, fe51eff673b24b2b8191a28d4d617fed; no voters:
I20260810 20:59:56.731604 20186 raft_consensus.cc:2781] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:56.731676 20186 raft_consensus.cc:482] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:56.731693 20186 raft_consensus.cc:3037] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:56.732374 20186 raft_consensus.cc:504] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.732520 20186 leader_election.cc:288] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [CANDIDATE]: Term 1 election: Requested vote from peers b78dfcb2b7a248caaf2c5d05cfab8328 (127.14.200.3:41239), d214d32265d440919892bbb18658a1fc (127.14.200.1:34459)
I20260810 20:59:56.732698 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c49ec84a0b59462eb0dbda67e3163c66" candidate_uuid: "fe51eff673b24b2b8191a28d4d617fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d214d32265d440919892bbb18658a1fc"
I20260810 20:59:56.732762 19813 raft_consensus.cc:3037] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:56.733479 19813 raft_consensus.cc:2454] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe51eff673b24b2b8191a28d4d617fed in term 1.
I20260810 20:59:56.733594 20110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c49ec84a0b59462eb0dbda67e3163c66" candidate_uuid: "fe51eff673b24b2b8191a28d4d617fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328"
I20260810 20:59:56.733628 19891 leader_election.cc:302] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d214d32265d440919892bbb18658a1fc, fe51eff673b24b2b8191a28d4d617fed; no voters:
I20260810 20:59:56.733669 20110 raft_consensus.cc:3037] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:56.733706 20186 raft_consensus.cc:2781] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:56.733876 20186 raft_consensus.cc:686] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 1 LEADER]: Becoming Leader. State: Replica: fe51eff673b24b2b8191a28d4d617fed, State: Running, Role: LEADER
I20260810 20:59:56.733979 20186 consensus_queue.cc:237] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } }
I20260810 20:59:56.734567 20110 raft_consensus.cc:2454] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe51eff673b24b2b8191a28d4d617fed in term 1.
I20260810 20:59:56.734917 19648 catalog_manager.cc:5495] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed reported cstate change: term changed from 0 to 1, leader changed from <none> to fe51eff673b24b2b8191a28d4d617fed (127.14.200.2). New cstate: current_term: 1 leader_uuid: "fe51eff673b24b2b8191a28d4d617fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe51eff673b24b2b8191a28d4d617fed" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46087 } health_report { overall_health: HEALTHY } } }
I20260810 20:59:56.746716 20180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.744646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.14.200.2:46087)
0810 20:59:56.744698 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.744700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.744719 (+ 19us) server_negotiation.cc:407] Connection header received
0810 20:59:56.744828 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.744829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.744846 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.744912 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 20:59:56.745050 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.745487 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.746121 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:56.746274 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:56.746317 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:56.746459 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 20:59:56.746561 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 20:59:56.746614 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.746643 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.746658 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 20:59:56.748273 20110 raft_consensus.cc:1261] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 1 FOLLOWER]: Refusing update from remote peer fe51eff673b24b2b8191a28d4d617fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:56.748274 19813 raft_consensus.cc:1261] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 1 FOLLOWER]: Refusing update from remote peer fe51eff673b24b2b8191a28d4d617fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:56.748584 20193 consensus_queue.cc:1035] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "b78dfcb2b7a248caaf2c5d05cfab8328" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 41239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:56.748684 20186 consensus_queue.cc:1035] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d214d32265d440919892bbb18658a1fc" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:56.750765 20198 mvcc.cc:205] Tried to move back new op lower bound from 7317076364278460416 to 7317076364223021056. Current Snapshot: MvccSnapshot[applied={T|T < 7317076364278460416}]
I20260810 20:59:56.750890 20200 mvcc.cc:205] Tried to move back new op lower bound from 7317076364278460416 to 7317076364223021056. Current Snapshot: MvccSnapshot[applied={T|T < 7317076364278460416}]
I20260810 20:59:56.750983 20201 mvcc.cc:205] Tried to move back new op lower bound from 7317076364278460416 to 7317076364223021056. Current Snapshot: MvccSnapshot[applied={T|T < 7317076364278460416}]
I20260810 20:59:56.753233 19647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:56.753336 19647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:56.754107 19647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdff3a0cb005493a982947898773665a: Sending DeleteTablet for 3 replicas of tablet c49ec84a0b59462eb0dbda67e3163c66
I20260810 20:59:56.754499 19790 tablet_service.cc:1514] Processing DeleteTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:56 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:40376
I20260810 20:59:56.754551 20088 tablet_service.cc:1514] Processing DeleteTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:56 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:57598
I20260810 20:59:56.754739 20209 tablet_replica.cc:330] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc: stopping tablet replica
I20260810 20:59:56.754839 20209 raft_consensus.cc:2229] T c49ec84a0b59462eb0dbda67e3163c66 P d214d32265d440919892bbb18658a1fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:56.754910 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19722
I20260810 20:59:56.763162 20210 tablet_replica.cc:330] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: stopping tablet replica
I20260810 20:59:56.763283 20210 raft_consensus.cc:2229] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:56.763371 19938 tablet_service.cc:1514] Processing DeleteTablet for tablet c49ec84a0b59462eb0dbda67e3163c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:56 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:46276
I20260810 20:59:56.763450 20210 raft_consensus.cc:2258] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:56.763600 20211 tablet_replica.cc:330] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: stopping tablet replica
I20260810 20:59:56.763875 20211 raft_consensus.cc:2229] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:56.763895 20210 ts_tablet_manager.cc:1905] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:56.764036 20211 raft_consensus.cc:2258] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:56.764520 20211 ts_tablet_manager.cc:1905] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:56.766098 20210 ts_tablet_manager.cc:1918] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:56.766217 20210 log.cc:1196] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/c49ec84a0b59462eb0dbda67e3163c66
I20260810 20:59:56.766258 20211 ts_tablet_manager.cc:1918] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:56.766443 20211 log.cc:1196] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/c49ec84a0b59462eb0dbda67e3163c66
I20260810 20:59:56.766584 20210 ts_tablet_manager.cc:1939] T c49ec84a0b59462eb0dbda67e3163c66 P b78dfcb2b7a248caaf2c5d05cfab8328: Deleting consensus metadata
I20260810 20:59:56.766803 20211 ts_tablet_manager.cc:1939] T c49ec84a0b59462eb0dbda67e3163c66 P fe51eff673b24b2b8191a28d4d617fed: Deleting consensus metadata
I20260810 20:59:56.767061 19631 catalog_manager.cc:4841] TS b78dfcb2b7a248caaf2c5d05cfab8328 (127.14.200.3:41239): tablet c49ec84a0b59462eb0dbda67e3163c66 (table test-table [id=4ad22c70c3914e8fb46677af8504ad8f]) successfully deleted
I20260810 20:59:56.767241 19634 catalog_manager.cc:4841] TS fe51eff673b24b2b8191a28d4d617fed (127.14.200.2:46087): tablet c49ec84a0b59462eb0dbda67e3163c66 (table test-table [id=4ad22c70c3914e8fb46677af8504ad8f]) successfully deleted
W20260810 20:59:56.768599 19633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:34459 (error 108)
I20260810 20:59:56.768960 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19872
I20260810 20:59:56.769131 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.768983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34459 (local address 127.0.0.1:40388)
0810 20:59:56.769048 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:56.769096 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:34459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 20:59:56.772677 19633 catalog_manager.cc:4584] TS d214d32265d440919892bbb18658a1fc (127.14.200.1:34459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c49ec84a0b59462eb0dbda67e3163c66: Network error: Client connection negotiation failed: client connection to 127.14.200.1:34459: connect: Connection refused (error 111)
I20260810 20:59:56.778997 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20017
I20260810 20:59:56.786374 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 19615
2026-08-10T20:59:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3587 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T20:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:56Z Disabled control of system clock
I20260810 20:59:56.815554 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41021
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 20:59:56.898165 20217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:56.898348 20217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:56.898367 20217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:56.899322 20217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:56.899353 20217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:56.899364 20217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:56.899375 20217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:56.899386 20217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:56.900569 20217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:56.900748 20217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:56.900987 20217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:56.905660 20223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.905715 20224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:56.905699 20226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:56.906069 20217 server_base.cc:1034] running on GCE node
I20260810 20:59:56.906260 20217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:56.906514 20217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:56.907660 20217 hybrid_clock.cc:648] HybridClock initialized: now 1786395596907650 us; error 27 us; skew 500 ppm
I20260810 20:59:56.908736 20217 webserver.cc:460] Webserver started at http://127.14.200.62:33741/ using document root <none> and password file <none>
I20260810 20:59:56.909009 20217 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:56.909053 20217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:56.909137 20217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:56.910125 20217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "040aa413008f4859b597c8e5246ed641"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "0427bc70ba1ef3094bfaa6434293a605"
server_key_iv: "dc7a3460aa1624c6ba690d52b104c6e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.910460 20217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "040aa413008f4859b597c8e5246ed641"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "0427bc70ba1ef3094bfaa6434293a605"
server_key_iv: "dc7a3460aa1624c6ba690d52b104c6e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.911693 20217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:56.912295 20232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:56.912469 20217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:56.912554 20217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "040aa413008f4859b597c8e5246ed641"
format_stamp: "Formatted at 2026-08-10 20:59:56 on dist-test-slave-1x4g"
server_key: "0427bc70ba1ef3094bfaa6434293a605"
server_key_iv: "dc7a3460aa1624c6ba690d52b104c6e0"
server_key_version: "encryptionkey@0"
I20260810 20:59:56.912606 20217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:56.915795 20217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:56.915975 20217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:56.916070 20217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:56.919895 20217 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41021
I20260810 20:59:56.919960 20289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41021 every 8 connection(s)
I20260810 20:59:56.920249 20217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:56.921006 20290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:56.923662 20290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: Bootstrap starting.
I20260810 20:59:56.924234 20290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:56.924568 20290 log.cc:824] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:56.925262 20290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: No bootstrap required, opened a new log
I20260810 20:59:56.926790 20290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } }
I20260810 20:59:56.926918 20290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:56.926983 20290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040aa413008f4859b597c8e5246ed641, State: Initialized, Role: FOLLOWER
I20260810 20:59:56.927161 20290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } }
I20260810 20:59:56.927238 20290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:56.927284 20290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:56.927328 20290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:56.927806 20290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } }
I20260810 20:59:56.927917 20290 leader_election.cc:302] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 040aa413008f4859b597c8e5246ed641; no voters:
I20260810 20:59:56.928088 20290 leader_election.cc:288] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:56.928121 20295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:56.928273 20295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [term 1 LEADER]: Becoming Leader. State: Replica: 040aa413008f4859b597c8e5246ed641, State: Running, Role: LEADER
I20260810 20:59:56.928326 20290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:56.928397 20295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } }
I20260810 20:59:56.928826 20298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 040aa413008f4859b597c8e5246ed641. Latest consensus state: current_term: 1 leader_uuid: "040aa413008f4859b597c8e5246ed641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } } }
I20260810 20:59:56.928934 20298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:56.929061 20297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "040aa413008f4859b597c8e5246ed641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040aa413008f4859b597c8e5246ed641" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41021 } } }
I20260810 20:59:56.929141 20297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:56.929558 20305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:56.930037 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20217
I20260810 20:59:56.930138 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:56.930335 15136 external_mini_cluster.cc:1428] Setting key 2e0d965a9034d92361d08c6968b98c2f
I20260810 20:59:56.930107 20305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:56.932017 20305 catalog_manager.cc:1347] Generated new cluster ID: 84f07a6c99ca424ea807206cdf7257b9
I20260810 20:59:56.932067 20305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:56.932964 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:56.930778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.14.200.62:41021)
0810 20:59:56.930973 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:56.930980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:56.931010 (+ 30us) server_negotiation.cc:407] Connection header received
0810 20:59:56.931163 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:56.931170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:56.931208 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:56.931342 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:56.932382 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:56.932385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:56.932386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:56.932389 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:56.932432 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:56.932444 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:56.932518 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:56.932614 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 20:59:56.932774 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260810 20:59:56.949755 20305 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:56.949930 20305 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:56.963804 20305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: Generated new TSK 0
I20260810 20:59:56.963995 20305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 20:59:56.974057 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--builtin_ntp_servers=127.14.200.20:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 20:59:57.055303 20319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:57.055505 20319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:57.055526 20319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:57.056691 20319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:57.056730 20319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:57.056756 20319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 20:59:57.058159 20319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:57.058398 20319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:57.058682 20319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:57.063616 20327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.063643 20325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.063648 20319 server_base.cc:1034] running on GCE node
W20260810 20:59:57.063616 20324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.064131 20319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:57.064399 20319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:57.065537 20319 hybrid_clock.cc:648] HybridClock initialized: now 1786395597065519 us; error 36 us; skew 500 ppm
I20260810 20:59:57.066669 20319 webserver.cc:460] Webserver started at http://127.14.200.1:43729/ using document root <none> and password file <none>
I20260810 20:59:57.066860 20319 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:57.066898 20319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:57.066985 20319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:57.067998 20319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "f3eb584747c54dee824fcb998294d70f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "fa2ee0d17028125c3a3f2706bb34ddd2"
server_key_iv: "a94ee23dcdef6adbe7ce6aa422e4045e"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.068365 20319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "f3eb584747c54dee824fcb998294d70f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "fa2ee0d17028125c3a3f2706bb34ddd2"
server_key_iv: "a94ee23dcdef6adbe7ce6aa422e4045e"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.069799 20319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:57.070443 20334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.070619 20319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:57.070708 20319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "f3eb584747c54dee824fcb998294d70f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "fa2ee0d17028125c3a3f2706bb34ddd2"
server_key_iv: "a94ee23dcdef6adbe7ce6aa422e4045e"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.070782 20319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:57.074126 20319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:57.074366 20319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:57.074504 20319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:57.074695 20319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:57.074978 20319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:57.075011 20319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.075062 20319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:57.075088 20319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.080631 20319 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:43347
I20260810 20:59:57.080705 20461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:43347 every 8 connection(s)
I20260810 20:59:57.081024 20319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 20:59:57.083846 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.081690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:57055 (local address 127.14.200.62:41021)
0810 20:59:57.081729 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.081732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.081913 (+ 181us) server_negotiation.cc:407] Connection header received
0810 20:59:57.082357 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.082359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.082369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.082426 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.083608 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.083610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.083610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.083615 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.083632 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.083636 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.083688 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.083763 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.083781 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 20:59:57.084025 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.081650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41021 (local address 127.14.200.1:57055)
0810 20:59:57.081807 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.081837 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.082257 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.082508 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.082520 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.082529 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.083453 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.083454 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.083479 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.083506 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.083684 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.083689 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.083693 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.083768 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.083827 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260810 20:59:57.084493 20462 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41021
I20260810 20:59:57.084573 20462 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:57.084740 20462 heartbeater.cc:511] Master 127.14.200.62:41021 requested a full tablet report, sending...
I20260810 20:59:57.085220 20251 ts_manager.cc:194] Registered new tserver with Master: f3eb584747c54dee824fcb998294d70f (127.14.200.1:43347)
I20260810 20:59:57.089459 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20319
I20260810 20:59:57.089541 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 20:59:57.089725 15136 external_mini_cluster.cc:1428] Setting key d004cafb5a02387610150d2c911ef7f8
I20260810 20:59:57.090664 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--builtin_ntp_servers=127.14.200.20:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 20:59:57.171793 20467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:57.171986 20467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:57.172008 20467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:57.173235 20467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:57.173286 20467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:57.173321 20467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 20:59:57.174805 20467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:57.174985 20467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:57.175216 20467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:57.180114 20473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.180162 20472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.180124 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.180498 20467 server_base.cc:1034] running on GCE node
I20260810 20:59:57.180699 20467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:57.180945 20467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:57.182082 20467 hybrid_clock.cc:648] HybridClock initialized: now 1786395597182065 us; error 34 us; skew 500 ppm
I20260810 20:59:57.183203 20467 webserver.cc:460] Webserver started at http://127.14.200.2:39619/ using document root <none> and password file <none>
I20260810 20:59:57.183416 20467 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:57.183465 20467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:57.183626 20467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:57.184736 20467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "7e44c1a649f243d883e78dc32275a76d"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "e2f91f05ff10c24f0c0c68887a68d810"
server_key_iv: "1de99e9d244189835dab2ea7f24c2ab8"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.185200 20467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "7e44c1a649f243d883e78dc32275a76d"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "e2f91f05ff10c24f0c0c68887a68d810"
server_key_iv: "1de99e9d244189835dab2ea7f24c2ab8"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.186594 20467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:57.187224 20483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.187383 20467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:57.187482 20467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "7e44c1a649f243d883e78dc32275a76d"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "e2f91f05ff10c24f0c0c68887a68d810"
server_key_iv: "1de99e9d244189835dab2ea7f24c2ab8"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.187557 20467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:57.190625 20467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:57.190868 20467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:57.191018 20467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:57.191220 20467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:57.191517 20467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:57.191581 20467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.191617 20467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:57.191641 20467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.196956 20467 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:37719
I20260810 20:59:57.197019 20613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:37719 every 8 connection(s)
I20260810 20:59:57.197324 20467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 20:59:57.200022 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.198126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39519 (local address 127.14.200.62:41021)
0810 20:59:57.198190 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.198192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.198258 (+ 66us) server_negotiation.cc:407] Connection header received
0810 20:59:57.198698 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.198699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.198703 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.198754 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.199847 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.199848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.199848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.199850 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.199861 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.199865 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.199903 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.199960 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.199971 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 20:59:57.200177 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.197974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41021 (local address 127.14.200.2:39519)
0810 20:59:57.198149 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.198184 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.198620 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.198799 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.198801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.198809 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.199728 (+ 919us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.199729 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.199755 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.199780 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.199902 (+ 122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.199905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.199907 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.199954 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.199996 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260810 20:59:57.200610 20614 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41021
I20260810 20:59:57.200701 20614 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:57.200848 20614 heartbeater.cc:511] Master 127.14.200.62:41021 requested a full tablet report, sending...
I20260810 20:59:57.201175 20251 ts_manager.cc:194] Registered new tserver with Master: 7e44c1a649f243d883e78dc32275a76d (127.14.200.2:37719)
I20260810 20:59:57.206054 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20467
I20260810 20:59:57.206130 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 20:59:57.206255 15136 external_mini_cluster.cc:1428] Setting key c8d3352fd53ae865262642a25042f23a
I20260810 20:59:57.207192 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--builtin_ntp_servers=127.14.200.20:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 20:59:57.287037 20617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:57.287245 20617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:57.287278 20617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:57.288362 20617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:57.288427 20617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:57.288481 20617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 20:59:57.289897 20617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:57.290140 20617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:57.290390 20617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:57.295122 20622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.295137 20624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.295265 20617 server_base.cc:1034] running on GCE node
W20260810 20:59:57.295111 20626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.295567 20617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:57.295763 20617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:57.296841 20617 hybrid_clock.cc:648] HybridClock initialized: now 1786395597296828 us; error 27 us; skew 500 ppm
I20260810 20:59:57.298034 20617 webserver.cc:460] Webserver started at http://127.14.200.3:40635/ using document root <none> and password file <none>
I20260810 20:59:57.298203 20617 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:57.298239 20617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:57.298321 20617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:57.299263 20617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "fa6b9a39b21549b190443c904d1e601f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "cc095f5da44736b5411d92474aae8b11"
server_key_iv: "28e1dcfee01f9d848d915f6179037c75"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.299582 20617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "fa6b9a39b21549b190443c904d1e601f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "cc095f5da44736b5411d92474aae8b11"
server_key_iv: "28e1dcfee01f9d848d915f6179037c75"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.300900 20617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:57.301611 20632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.301761 20617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:57.301846 20617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "fa6b9a39b21549b190443c904d1e601f"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "cc095f5da44736b5411d92474aae8b11"
server_key_iv: "28e1dcfee01f9d848d915f6179037c75"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.301898 20617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:57.305150 20617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:57.305342 20617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:57.305429 20617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:57.305613 20617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 20:59:57.305837 20617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 20:59:57.305866 20617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.305886 20617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 20:59:57.305897 20617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.311372 20617 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:38747
I20260810 20:59:57.311439 20762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:38747 every 8 connection(s)
I20260810 20:59:57.311730 20617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 20:59:57.312311 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20617
I20260810 20:59:57.312415 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 20:59:57.312646 15136 external_mini_cluster.cc:1428] Setting key e62375778e6d1c9f6b37b86d6084a13b
I20260810 20:59:57.314781 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.312434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:57825 (local address 127.14.200.62:41021)
0810 20:59:57.312636 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.312637 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.312789 (+ 152us) server_negotiation.cc:407] Connection header received
0810 20:59:57.313406 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.313407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.313413 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.313470 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.314583 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.314584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.314585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.314588 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.314603 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.314607 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.314653 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.314718 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.314730 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260810 20:59:57.314952 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.312438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41021 (local address 127.14.200.3:57825)
0810 20:59:57.312652 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.312693 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.313302 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.313534 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.313537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.313546 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.314420 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.314420 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.314445 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.314470 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.314655 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.314658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.314661 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.314712 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.314758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260810 20:59:57.315400 20764 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41021
I20260810 20:59:57.315495 20764 heartbeater.cc:464] Registering TS with master...
I20260810 20:59:57.315647 20764 heartbeater.cc:511] Master 127.14.200.62:41021 requested a full tablet report, sending...
I20260810 20:59:57.315951 20251 ts_manager.cc:194] Registered new tserver with Master: fa6b9a39b21549b190443c904d1e601f (127.14.200.3:38747)
I20260810 20:59:57.324090 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 20:59:57.326417 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.325580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.14.200.62:41021)
0810 20:59:57.325633 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.325635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.325725 (+ 90us) server_negotiation.cc:407] Connection header received
0810 20:59:57.325763 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.325764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.325768 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.325825 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.326272 (+ 447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.326273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.326273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.326276 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.326288 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.326292 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.326335 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.326366 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.326377 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 20:59:57.327772 20251 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 20:59:57.329403 20251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 20:59:57.333178 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:38747 (local address 127.0.0.1:59678)
0810 20:59:57.331677 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.331716 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.331772 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.331965 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.331971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.331996 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.332504 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.332505 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.332521 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.332544 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.333009 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.333016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.333019 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.333091 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.333106 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":63,"threads_started":1}
I20260810 20:59:57.333231 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.14.200.3:38747)
0810 20:59:57.331847 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.331852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.331861 (+ 9us) server_negotiation.cc:407] Connection header received
0810 20:59:57.331881 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.331883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.331890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.331956 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.332834 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.332842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.332842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.332846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.332880 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.332901 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.332979 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.333108 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.333124 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260810 20:59:57.333675 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43347 (local address 127.0.0.1:51932)
0810 20:59:57.331686 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.331718 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.331775 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.332623 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.332623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.332626 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.333064 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.333064 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.333075 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.333089 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.333544 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.333545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.333546 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.333605 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.333618 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260810 20:59:57.333705 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.14.200.1:43347)
0810 20:59:57.332465 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.332469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.332480 (+ 11us) server_negotiation.cc:407] Connection header received
0810 20:59:57.332509 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.332511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.332520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.332605 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.333385 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.333392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.333393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.333397 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.333432 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.333460 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.333542 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.333612 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.333627 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":131,"threads_started":1}
I20260810 20:59:57.333828 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:37719 (local address 127.0.0.1:50928)
0810 20:59:57.332181 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.332191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.332236 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.332714 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.332715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.332717 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.333210 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.333210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.333215 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.333224 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.333662 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.333663 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.333663 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.333704 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.333718 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":138,"thread_start_us":71,"threads_started":1}
I20260810 20:59:57.333901 20684 tablet_service.cc:1467] Processing CreateTablet for tablet be3b7b97965f4b94a27bea04048c77a2 (DEFAULT_TABLE table=test-table [id=bf40770a64294a9d8958a72ce2810074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:57.334299 20684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b7b97965f4b94a27bea04048c77a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:57.334300 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.331565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50928 (local address 127.14.200.2:37719)
0810 20:59:57.331910 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.331915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.332295 (+ 380us) server_negotiation.cc:407] Connection header received
0810 20:59:57.332325 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.332328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.332337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.332404 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.333507 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.333514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.333515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.333519 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.333551 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.333577 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.333635 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.333891 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.333910 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":100,"threads_started":1}
I20260810 20:59:57.334885 20534 tablet_service.cc:1467] Processing CreateTablet for tablet be3b7b97965f4b94a27bea04048c77a2 (DEFAULT_TABLE table=test-table [id=bf40770a64294a9d8958a72ce2810074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:57.335193 20534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b7b97965f4b94a27bea04048c77a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:57.337152 20779 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Bootstrap starting.
I20260810 20:59:57.337185 20390 tablet_service.cc:1467] Processing CreateTablet for tablet be3b7b97965f4b94a27bea04048c77a2 (DEFAULT_TABLE table=test-table [id=bf40770a64294a9d8958a72ce2810074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 20:59:57.337527 20390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b7b97965f4b94a27bea04048c77a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:57.337915 20779 tablet_bootstrap.cc:654] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:57.338053 20780 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Bootstrap starting.
I20260810 20:59:57.338342 20779 log.cc:824] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:57.338786 20780 tablet_bootstrap.cc:654] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:57.339077 20780 log.cc:824] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:57.339200 20779 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: No bootstrap required, opened a new log
I20260810 20:59:57.339273 20779 ts_tablet_manager.cc:1397] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:57.339812 20780 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: No bootstrap required, opened a new log
I20260810 20:59:57.339902 20780 ts_tablet_manager.cc:1397] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 20:59:57.340634 20783 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: Bootstrap starting.
I20260810 20:59:57.341384 20783 tablet_bootstrap.cc:654] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:57.341711 20783 log.cc:824] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:57.341785 20779 raft_consensus.cc:348] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.342056 20779 raft_consensus.cc:374] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:57.342154 20779 raft_consensus.cc:729] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6b9a39b21549b190443c904d1e601f, State: Initialized, Role: FOLLOWER
I20260810 20:59:57.342315 20779 consensus_queue.cc:260] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.342365 20780 raft_consensus.cc:348] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.342557 20783 tablet_bootstrap.cc:492] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: No bootstrap required, opened a new log
I20260810 20:59:57.342603 20764 heartbeater.cc:503] Master 127.14.200.62:41021 was elected leader, sending a full tablet report...
I20260810 20:59:57.342592 20780 raft_consensus.cc:374] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:57.342651 20780 raft_consensus.cc:729] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e44c1a649f243d883e78dc32275a76d, State: Initialized, Role: FOLLOWER
I20260810 20:59:57.342712 20783 ts_tablet_manager.cc:1397] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 20:59:57.342772 20780 consensus_queue.cc:260] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.342859 20779 ts_tablet_manager.cc:1428] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 20:59:57.343046 20780 ts_tablet_manager.cc:1428] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 20:59:57.343083 20614 heartbeater.cc:503] Master 127.14.200.62:41021 was elected leader, sending a full tablet report...
I20260810 20:59:57.345405 20783 raft_consensus.cc:348] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.345536 20783 raft_consensus.cc:374] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:57.345556 20783 raft_consensus.cc:729] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3eb584747c54dee824fcb998294d70f, State: Initialized, Role: FOLLOWER
I20260810 20:59:57.345683 20783 consensus_queue.cc:260] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.345925 20783 ts_tablet_manager.cc:1428] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 20:59:57.345990 20462 heartbeater.cc:503] Master 127.14.200.62:41021 was elected leader, sending a full tablet report...
W20260810 20:59:57.447889 20615 tablet.cc:2369] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 20:59:57.510825 20786 raft_consensus.cc:482] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 20:59:57.510996 20786 raft_consensus.cc:504] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.511371 20786 leader_election.cc:288] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa6b9a39b21549b190443c904d1e601f (127.14.200.3:38747), f3eb584747c54dee824fcb998294d70f (127.14.200.1:43347)
I20260810 20:59:57.512704 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.511555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:38747 (local address 127.14.200.2:54369)
0810 20:59:57.511595 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.511606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.511643 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.511806 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.511808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.511811 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.512272 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.512273 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.512281 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.512296 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.512526 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.512527 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.512528 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.512612 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.512625 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 20:59:57.512732 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.511617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54369 (local address 127.14.200.3:38747)
0810 20:59:57.511661 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.511663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.511671 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:57.511703 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.511705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.511711 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.511784 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.512396 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.512399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.512400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.512409 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.512449 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.512455 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.512517 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.512619 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.512633 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 20:59:57.512850 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.511665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:36561 (local address 127.14.200.1:43347)
0810 20:59:57.511725 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.511727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.511954 (+ 227us) server_negotiation.cc:407] Connection header received
0810 20:59:57.512025 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.512026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.512031 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.512077 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.512601 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.512603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.512603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.512608 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.512628 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.512633 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.512684 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.512741 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.512755 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 20:59:57.512871 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.511555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43347 (local address 127.14.200.2:36561)
0810 20:59:57.511887 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.511896 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.511963 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.512079 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.512080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.512082 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 20:59:57.512480 (+ 398us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 20:59:57.512481 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 20:59:57.512488 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 20:59:57.512502 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 20:59:57.512692 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 20:59:57.512693 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 20:59:57.512693 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 20:59:57.512741 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 20:59:57.512754 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":133,"thread_start_us":118,"threads_started":1}
I20260810 20:59:57.513226 20411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b7b97965f4b94a27bea04048c77a2" candidate_uuid: "7e44c1a649f243d883e78dc32275a76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3eb584747c54dee824fcb998294d70f" is_pre_election: true
I20260810 20:59:57.513248 20709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b7b97965f4b94a27bea04048c77a2" candidate_uuid: "7e44c1a649f243d883e78dc32275a76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa6b9a39b21549b190443c904d1e601f" is_pre_election: true
I20260810 20:59:57.513372 20411 raft_consensus.cc:2454] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e44c1a649f243d883e78dc32275a76d in term 0.
I20260810 20:59:57.513382 20709 raft_consensus.cc:2454] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e44c1a649f243d883e78dc32275a76d in term 0.
I20260810 20:59:57.513620 20484 leader_election.cc:302] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e44c1a649f243d883e78dc32275a76d, fa6b9a39b21549b190443c904d1e601f; no voters:
I20260810 20:59:57.513751 20786 raft_consensus.cc:2781] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 20:59:57.513803 20786 raft_consensus.cc:482] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 20:59:57.513836 20786 raft_consensus.cc:3037] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:57.514638 20786 raft_consensus.cc:504] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.514786 20786 leader_election.cc:288] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [CANDIDATE]: Term 1 election: Requested vote from peers fa6b9a39b21549b190443c904d1e601f (127.14.200.3:38747), f3eb584747c54dee824fcb998294d70f (127.14.200.1:43347)
I20260810 20:59:57.514900 20709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b7b97965f4b94a27bea04048c77a2" candidate_uuid: "7e44c1a649f243d883e78dc32275a76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa6b9a39b21549b190443c904d1e601f"
I20260810 20:59:57.514966 20411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b7b97965f4b94a27bea04048c77a2" candidate_uuid: "7e44c1a649f243d883e78dc32275a76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3eb584747c54dee824fcb998294d70f"
I20260810 20:59:57.514994 20709 raft_consensus.cc:3037] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:57.515034 20411 raft_consensus.cc:3037] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:57.515940 20709 raft_consensus.cc:2454] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e44c1a649f243d883e78dc32275a76d in term 1.
I20260810 20:59:57.515933 20411 raft_consensus.cc:2454] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e44c1a649f243d883e78dc32275a76d in term 1.
I20260810 20:59:57.516104 20484 leader_election.cc:302] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e44c1a649f243d883e78dc32275a76d, fa6b9a39b21549b190443c904d1e601f; no voters:
I20260810 20:59:57.516216 20786 raft_consensus.cc:2781] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:57.516389 20786 raft_consensus.cc:686] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 1 LEADER]: Becoming Leader. State: Replica: 7e44c1a649f243d883e78dc32275a76d, State: Running, Role: LEADER
I20260810 20:59:57.516523 20786 consensus_queue.cc:237] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } }
I20260810 20:59:57.517351 20250 catalog_manager.cc:5495] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e44c1a649f243d883e78dc32275a76d (127.14.200.2). New cstate: current_term: 1 leader_uuid: "7e44c1a649f243d883e78dc32275a76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e44c1a649f243d883e78dc32275a76d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 37719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 } health_report { overall_health: UNKNOWN } } }
I20260810 20:59:57.559078 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.558209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.14.200.2:37719)
0810 20:59:57.558272 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.558274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.558282 (+ 8us) server_negotiation.cc:407] Connection header received
0810 20:59:57.558322 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.558323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.558338 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.558385 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.558889 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.558891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.558891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.558894 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.558910 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.558914 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.558971 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.559016 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.559028 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 20:59:57.560887 20709 raft_consensus.cc:1261] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 1 FOLLOWER]: Refusing update from remote peer 7e44c1a649f243d883e78dc32275a76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:57.560894 20411 raft_consensus.cc:1261] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 1 FOLLOWER]: Refusing update from remote peer 7e44c1a649f243d883e78dc32275a76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 20:59:57.561230 20786 consensus_queue.cc:1035] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6b9a39b21549b190443c904d1e601f" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 20:59:57.561370 20789 consensus_queue.cc:1035] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3eb584747c54dee824fcb998294d70f" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 20:59:57.562225 20765 tablet.cc:2369] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 20:59:57.563730 20793 mvcc.cc:205] Tried to move back new op lower bound from 7317076367606636544 to 7317076367428247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317076367606636544}]
I20260810 20:59:57.564157 20795 mvcc.cc:205] Tried to move back new op lower bound from 7317076367606636544 to 7317076367428247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317076367606636544}]
I20260810 20:59:57.564175 20794 mvcc.cc:205] Tried to move back new op lower bound from 7317076367606636544 to 7317076367428247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317076367606636544}]
I20260810 20:59:57.566370 20251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:57.566483 20251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 20:59:57.567227 20251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 040aa413008f4859b597c8e5246ed641: Sending DeleteTablet for 3 replicas of tablet be3b7b97965f4b94a27bea04048c77a2
I20260810 20:59:57.567564 20390 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b7b97965f4b94a27bea04048c77a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:57 UTC) from {username='slave'} at 127.0.0.1:51932
I20260810 20:59:57.567608 20684 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b7b97965f4b94a27bea04048c77a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:57 UTC) from {username='slave'} at 127.0.0.1:59678
I20260810 20:59:57.567764 20803 tablet_replica.cc:330] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f: stopping tablet replica
I20260810 20:59:57.567814 20804 tablet_replica.cc:330] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: stopping tablet replica
I20260810 20:59:57.567811 20534 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b7b97965f4b94a27bea04048c77a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 20:59:57 UTC) from {username='slave'} at 127.0.0.1:50928
I20260810 20:59:57.567953 20803 raft_consensus.cc:2229] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:57.567972 20804 raft_consensus.cc:2229] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 20:59:57.568073 20804 raft_consensus.cc:2258] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:57.568133 20803 raft_consensus.cc:2258] T be3b7b97965f4b94a27bea04048c77a2 P f3eb584747c54dee824fcb998294d70f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:57.568192 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20319
I20260810 20:59:57.569890 20804 ts_tablet_manager.cc:1905] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:57.572368 20804 ts_tablet_manager.cc:1918] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:57.572487 20804 log.cc:1196] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/be3b7b97965f4b94a27bea04048c77a2
I20260810 20:59:57.572865 20804 ts_tablet_manager.cc:1939] T be3b7b97965f4b94a27bea04048c77a2 P fa6b9a39b21549b190443c904d1e601f: Deleting consensus metadata
I20260810 20:59:57.577023 20233 catalog_manager.cc:4841] TS fa6b9a39b21549b190443c904d1e601f (127.14.200.3:38747): tablet be3b7b97965f4b94a27bea04048c77a2 (table test-table [id=bf40770a64294a9d8958a72ce2810074]) successfully deleted
I20260810 20:59:57.578256 20805 tablet_replica.cc:330] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: stopping tablet replica
I20260810 20:59:57.578382 20805 raft_consensus.cc:2229] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 1 LEADER]: Raft consensus shutting down.
I20260810 20:59:57.578514 20805 raft_consensus.cc:2258] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 20:59:57.578953 20805 ts_tablet_manager.cc:1905] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 20:59:57.580457 20805 ts_tablet_manager.cc:1918] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 20:59:57.580540 20805 log.cc:1196] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/be3b7b97965f4b94a27bea04048c77a2
I20260810 20:59:57.580817 20805 ts_tablet_manager.cc:1939] T be3b7b97965f4b94a27bea04048c77a2 P 7e44c1a649f243d883e78dc32275a76d: Deleting consensus metadata
I20260810 20:59:57.581184 20234 catalog_manager.cc:4841] TS 7e44c1a649f243d883e78dc32275a76d (127.14.200.2:37719): tablet be3b7b97965f4b94a27bea04048c77a2 (table test-table [id=bf40770a64294a9d8958a72ce2810074]) successfully deleted
W20260810 20:59:57.583942 20235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:43347 (error 108)
I20260810 20:59:57.584159 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20467
I20260810 20:59:57.585269 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.585135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:43347 (local address 127.0.0.1:51946)
0810 20:59:57.585186 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.585232 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 20:59:57.585404 20235 catalog_manager.cc:4584] TS f3eb584747c54dee824fcb998294d70f (127.14.200.1:43347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be3b7b97965f4b94a27bea04048c77a2: Network error: Client connection negotiation failed: client connection to 127.14.200.1:43347: connect: Connection refused (error 111)
I20260810 20:59:57.591051 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20617
I20260810 20:59:57.597555 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20217
2026-08-10T20:59:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (810 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 20:59:57.617959 15136 mini_oidc.cc:152] Starting JWKS server
I20260810 20:59:57.619082 15136 webserver.cc:460] Webserver started at https://127.0.0.1:34847/ using document root <none> and password file <none>
I20260810 20:59:57.619159 15136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 20:59:57.619287 15136 webserver.cc:460] Webserver started at http://127.0.0.1:39515/ using document root <none> and password file <none>
2026-08-10T20:59:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:57Z Disabled control of system clock
I20260810 20:59:57.627923 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33931
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:44583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:33931
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34847/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/testchainca.cert with env {}
W20260810 20:59:57.706528 20814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:57.706709 20814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:57.706729 20814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:57.707792 20814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:57.707830 20814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:57.707842 20814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:57.707854 20814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:57.707866 20814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 20:59:57.707878 20814 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 20:59:57.707890 20814 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34847/jwks.json
I20260810 20:59:57.709357 20814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:33931
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34847/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:57.709538 20814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:57.709697 20814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:57.714267 20821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.714262 20823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.714278 20820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.714545 20814 server_base.cc:1034] running on GCE node
I20260810 20:59:57.714862 20814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:57.715147 20814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:57.716301 20814 hybrid_clock.cc:648] HybridClock initialized: now 1786395597716289 us; error 28 us; skew 500 ppm
I20260810 20:59:57.717557 20814 webserver.cc:460] Webserver started at http://127.14.200.62:40899/ using document root <none> and password file <none>
I20260810 20:59:57.717799 20814 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:57.717856 20814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:57.718021 20814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:57.719117 20814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "a9644b38d0ae4ca6b479ffba9659330b"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "7689b1a30d7a308acf2d67c836039c98"
server_key_iv: "25304e29b4ac5432be2ea0a0e5ba878b"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.719507 20814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "a9644b38d0ae4ca6b479ffba9659330b"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "7689b1a30d7a308acf2d67c836039c98"
server_key_iv: "25304e29b4ac5432be2ea0a0e5ba878b"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.720868 20814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 20:59:57.721558 20830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.721742 20814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:57.721835 20814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "a9644b38d0ae4ca6b479ffba9659330b"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "7689b1a30d7a308acf2d67c836039c98"
server_key_iv: "25304e29b4ac5432be2ea0a0e5ba878b"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.721908 20814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:57.729396 20814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:57.729660 20814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:57.729806 20814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:57.733652 20814 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:33931
I20260810 20:59:57.733681 20888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:33931 every 8 connection(s)
I20260810 20:59:57.733984 20814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:57.734985 20889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:57.737284 20889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: Bootstrap starting.
I20260810 20:59:57.737911 20889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:57.738209 20889 log.cc:824] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:57.738844 20889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: No bootstrap required, opened a new log
I20260810 20:59:57.740379 20889 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } }
I20260810 20:59:57.740516 20889 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:57.740561 20889 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9644b38d0ae4ca6b479ffba9659330b, State: Initialized, Role: FOLLOWER
I20260810 20:59:57.740703 20889 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } }
I20260810 20:59:57.740763 20889 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:57.740856 20889 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:57.740903 20889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:57.741429 20889 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } }
I20260810 20:59:57.741567 20889 leader_election.cc:302] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9644b38d0ae4ca6b479ffba9659330b; no voters:
I20260810 20:59:57.741729 20889 leader_election.cc:288] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:57.741829 20892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:57.741940 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20814
I20260810 20:59:57.742000 20892 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [term 1 LEADER]: Becoming Leader. State: Replica: a9644b38d0ae4ca6b479ffba9659330b, State: Running, Role: LEADER
I20260810 20:59:57.742036 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:57.742035 20889 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:57.742101 20892 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } }
I20260810 20:59:57.742254 15136 external_mini_cluster.cc:1428] Setting key 5ca39b8927501aa0e5074de21c29b6b2
I20260810 20:59:57.742452 20892 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9644b38d0ae4ca6b479ffba9659330b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } } }
I20260810 20:59:57.742506 20892 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:57.742744 20893 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9644b38d0ae4ca6b479ffba9659330b. Latest consensus state: current_term: 1 leader_uuid: "a9644b38d0ae4ca6b479ffba9659330b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9644b38d0ae4ca6b479ffba9659330b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33931 } } }
I20260810 20:59:57.742803 20893 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b [sys.catalog]: This master's current role is: LEADER
W20260810 20:59:57.743858 20911 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 20:59:57.743965 20911 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 20:59:57.744285 20898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:57.744516 20898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:57.746627 20898 catalog_manager.cc:1347] Generated new cluster ID: 105e45c953314a4ca7ffc0276333de40
I20260810 20:59:57.746677 20898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:57.748449 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.742824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.14.200.62:33931)
0810 20:59:57.743042 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.743050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.743332 (+ 282us) server_negotiation.cc:407] Connection header received
0810 20:59:57.743517 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.743526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.743618 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.744028 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.744119 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.745488 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.746297 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.746944 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.747089 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.747750 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.747754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.747754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.747961 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.748002 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.748028 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.748108 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.748167 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.748245 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260810 20:59:57.749379 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 20:59:57.751941 20898 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:57.752451 20898 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:57.755060 20898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9644b38d0ae4ca6b479ffba9659330b: Generated new TSK 0
I20260810 20:59:57.755219 20898 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 20:59:57.777364 20923 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 20:59:57.777503 20923 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 20:59:57.777715 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.762307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.14.200.62:33931)
0810 20:59:57.762358 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.762360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.762452 (+ 92us) server_negotiation.cc:407] Connection header received
0810 20:59:57.762520 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.762521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.762533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.762602 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0810 20:59:57.762737 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.763465 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.764177 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.764490 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.764558 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.764730 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 20:59:57.777504 (+ 12774us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 20:59:57.777651 (+ 147us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43666: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":32}
W20260810 20:59:57.777770 20904 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43666: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 20:59:57.778071 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 20:59:57.762213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:33931 (local address 127.0.0.1:43666)
0810 20:59:57.762395 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.762403 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.762457 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.762613 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.762614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.762616 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 20:59:57.762678 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:57.762679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.763551 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.763552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:57.764101 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:57.764102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.764565 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.764566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:57.764611 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.764630 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 20:59:57.777618 (+ 12988us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 20:59:57.777732 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.200.62:33931: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.200.62:33931: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34847/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 20:59:57.778532 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20814
2026-08-10T20:59:57Z chronyd exiting
I20260810 20:59:57.819895 15136 test_util.cc:175] -----------------------------------------------
I20260810 20:59:57.819973 15136 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786395574307397-15136-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 20:59:57.820719 15136 mini_oidc.cc:152] Starting JWKS server
I20260810 20:59:57.821416 15136 webserver.cc:460] Webserver started at https://127.0.0.1:46089/ using document root <none> and password file <none>
I20260810 20:59:57.821465 15136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 20:59:57.821569 15136 webserver.cc:460] Webserver started at http://127.0.0.1:36045/ using document root <none> and password file <none>
2026-08-10T20:59:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:57Z Disabled control of system clock
I20260810 20:59:57.830292 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42481
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:44781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:42481
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46089/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/testchainca.cert with env {}
W20260810 20:59:57.909298 20930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:57.909482 20930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:57.909504 20930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:57.910535 20930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:57.910566 20930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:57.910578 20930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:57.910590 20930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:57.910602 20930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 20:59:57.910614 20930 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 20:59:57.910624 20930 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46089/jwks.json
I20260810 20:59:57.912016 20930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:42481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46089/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:57.912194 20930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:57.912344 20930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:57.916918 20940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.916936 20937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:57.916934 20938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:57.917193 20930 server_base.cc:1034] running on GCE node
I20260810 20:59:57.917418 20930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:57.917688 20930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:57.918838 20930 hybrid_clock.cc:648] HybridClock initialized: now 1786395597918830 us; error 56 us; skew 500 ppm
I20260810 20:59:57.919960 20930 webserver.cc:460] Webserver started at http://127.14.200.62:42695/ using document root <none> and password file <none>
I20260810 20:59:57.920187 20930 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:57.920228 20930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:57.920377 20930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:57.921391 20930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "241cf3312d1d482cb13540c7889735d0"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "f2fe3cf158ea5c11f717d1667a7b3abf"
server_key_iv: "3917f55a5f8acaa443d769b3d4f569df"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.921799 20930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "241cf3312d1d482cb13540c7889735d0"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "f2fe3cf158ea5c11f717d1667a7b3abf"
server_key_iv: "3917f55a5f8acaa443d769b3d4f569df"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.923116 20930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 20:59:57.923835 20947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:57.924011 20930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 20:59:57.924115 20930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "241cf3312d1d482cb13540c7889735d0"
format_stamp: "Formatted at 2026-08-10 20:59:57 on dist-test-slave-1x4g"
server_key: "f2fe3cf158ea5c11f717d1667a7b3abf"
server_key_iv: "3917f55a5f8acaa443d769b3d4f569df"
server_key_version: "encryptionkey@0"
I20260810 20:59:57.924198 20930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:57.946048 20930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:57.946338 20930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:57.946503 20930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:57.950454 20930 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:42481
I20260810 20:59:57.950484 21012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:42481 every 8 connection(s)
I20260810 20:59:57.950815 20930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:57.951493 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:57.954388 21016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0: Bootstrap starting.
I20260810 20:59:57.954680 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 20930
I20260810 20:59:57.954759 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:57.954950 15136 external_mini_cluster.cc:1428] Setting key d8d416db72c0763bdd3dfb4c50511095
I20260810 20:59:57.955176 21016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:57.955549 21016 log.cc:824] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:57.956336 21016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0: No bootstrap required, opened a new log
I20260810 20:59:57.958719 21016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } }
I20260810 20:59:57.958846 21016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:57.958868 21016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241cf3312d1d482cb13540c7889735d0, State: Initialized, Role: FOLLOWER
I20260810 20:59:57.959028 21016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } }
I20260810 20:59:57.959105 21016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:57.959134 21016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:57.959177 21016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:57.959842 21016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } }
I20260810 20:59:57.959981 21016 leader_election.cc:302] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 241cf3312d1d482cb13540c7889735d0; no voters:
I20260810 20:59:57.960134 21016 leader_election.cc:288] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:57.960182 21022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:57.960325 21022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [term 1 LEADER]: Becoming Leader. State: Replica: 241cf3312d1d482cb13540c7889735d0, State: Running, Role: LEADER
I20260810 20:59:57.960386 21016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:57.960418 21022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } }
I20260810 20:59:57.960757 21022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 241cf3312d1d482cb13540c7889735d0. Latest consensus state: current_term: 1 leader_uuid: "241cf3312d1d482cb13540c7889735d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } } }
I20260810 20:59:57.960868 21022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:57.961094 21031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:57.961298 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.955486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.14.200.62:42481)
0810 20:59:57.955722 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.955733 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.955769 (+ 36us) server_negotiation.cc:407] Connection header received
0810 20:59:57.956348 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.956357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.956431 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.956585 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:57.956814 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.958166 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.958928 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.959534 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.959668 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.960387 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:57.960390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:57.960390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:57.960591 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:57.960632 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:57.960647 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:57.960728 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:57.960979 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 20:59:57.961070 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
I20260810 20:59:57.961647 21023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "241cf3312d1d482cb13540c7889735d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241cf3312d1d482cb13540c7889735d0" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42481 } } }
I20260810 20:59:57.961757 21023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:57.961827 21031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:57.962201 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 20:59:57.963459 21031 catalog_manager.cc:1347] Generated new cluster ID: 5fc8144da7564b1cac3f989fe2dc9d5e
I20260810 20:59:57.963517 21031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:57.974439 21031 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:57.975173 21031 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:57.982786 21031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 241cf3312d1d482cb13540c7889735d0: Generated new TSK 0
I20260810 20:59:57.982954 21031 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 20:59:58.002173 21050 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 20:59:58.002245 21050 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 20:59:58.002568 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:57.988824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.14.200.62:42481)
0810 20:59:57.988865 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:57.988867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:57.988923 (+ 56us) server_negotiation.cc:407] Connection header received
0810 20:59:57.988993 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:57.988993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:57.989004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:57.989082 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0810 20:59:57.989211 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.990018 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.990754 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.991038 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.991103 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.991244 (+ 141us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 20:59:58.002356 (+ 11112us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46089/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 20:59:58.002491 (+ 135us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46089/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 20:59:58.002624 21020 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46089/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 20:59:58.002643 21049 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 20:59:57.988727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42481 (local address 127.0.0.1:60516)
0810 20:59:57.988867 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:57.988876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:57.988938 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:57.989094 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:57.989094 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:57.989096 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 20:59:57.989142 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:57.989144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.990107 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.990108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:57.990658 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:57.990660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:57.991117 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:57.991118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:57.991156 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:57.991160 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 20:59:58.002460 (+ 11300us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46089/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 20:59:58.002552 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.200.62:42481: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46089/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260810 20:59:58.002854 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 20930
2026-08-10T20:59:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 20:59:58.042863 15136 mini_oidc.cc:152] Starting JWKS server
I20260810 20:59:58.043503 15136 webserver.cc:460] Webserver started at https://127.0.0.1:41351/ using document root <none> and password file <none>
I20260810 20:59:58.043552 15136 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 20:59:58.043690 15136 webserver.cc:460] Webserver started at http://127.0.0.1:42913/ using document root <none> and password file <none>
2026-08-10T20:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:58Z Disabled control of system clock
I20260810 20:59:58.052241 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43075
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:43075
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41351/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 20:59:58.131932 21056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:58.132148 21056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:58.132180 21056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:58.133302 21056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:58.133365 21056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:58.133386 21056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:58.133407 21056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:58.133436 21056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 20:59:58.133459 21056 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 20:59:58.133488 21056 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41351/jwks.json
I20260810 20:59:58.134863 21056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:43075
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41351/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:58.135142 21056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:58.135429 21056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:58.140103 21065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:58.140110 21063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:58.140184 21062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:58.140347 21056 server_base.cc:1034] running on GCE node
I20260810 20:59:58.140601 21056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:58.140909 21056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:58.142066 21056 hybrid_clock.cc:648] HybridClock initialized: now 1786395598142051 us; error 26 us; skew 500 ppm
I20260810 20:59:58.143321 21056 webserver.cc:460] Webserver started at http://127.14.200.62:46621/ using document root <none> and password file <none>
I20260810 20:59:58.143595 21056 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:58.143666 21056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:58.143821 21056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:58.145076 21056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "6d79a47eca484af6a9e8616e1af53eec"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "96dc7018aa075dbc04107277558824f8"
server_key_iv: "03abc56559c800c1d51e28b6588dd4c8"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.145486 21056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "6d79a47eca484af6a9e8616e1af53eec"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "96dc7018aa075dbc04107277558824f8"
server_key_iv: "03abc56559c800c1d51e28b6588dd4c8"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.146943 21056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 20:59:58.147642 21071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:58.147807 21056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:58.147913 21056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "6d79a47eca484af6a9e8616e1af53eec"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "96dc7018aa075dbc04107277558824f8"
server_key_iv: "03abc56559c800c1d51e28b6588dd4c8"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.147987 21056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:58.171247 21056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:58.171618 21056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:58.171765 21056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:58.176214 21056 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:43075
I20260810 20:59:58.176265 21129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:43075 every 8 connection(s)
I20260810 20:59:58.176648 21056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:58.177371 21130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:58.180069 21130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec: Bootstrap starting.
I20260810 20:59:58.180626 21130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:58.180928 21130 log.cc:824] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:58.181540 21130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec: No bootstrap required, opened a new log
I20260810 20:59:58.182979 21130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } }
I20260810 20:59:58.183118 21130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:58.183141 21130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d79a47eca484af6a9e8616e1af53eec, State: Initialized, Role: FOLLOWER
I20260810 20:59:58.183269 21130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } }
I20260810 20:59:58.183351 21130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:58.183373 21130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:58.183400 21130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:58.183907 21130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } }
I20260810 20:59:58.183987 21130 leader_election.cc:302] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d79a47eca484af6a9e8616e1af53eec; no voters:
I20260810 20:59:58.184142 21130 leader_election.cc:288] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:58.184171 21133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:58.184281 21133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [term 1 LEADER]: Becoming Leader. State: Replica: 6d79a47eca484af6a9e8616e1af53eec, State: Running, Role: LEADER
I20260810 20:59:58.184413 21130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:58.184397 21133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } }
I20260810 20:59:58.184852 21135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d79a47eca484af6a9e8616e1af53eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } } }
I20260810 20:59:58.184968 21135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:58.185081 21136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d79a47eca484af6a9e8616e1af53eec. Latest consensus state: current_term: 1 leader_uuid: "6d79a47eca484af6a9e8616e1af53eec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d79a47eca484af6a9e8616e1af53eec" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43075 } } }
I20260810 20:59:58.185173 21136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:58.185611 21143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:58.186072 21143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:58.186774 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21056
I20260810 20:59:58.186872 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:58.187069 15136 external_mini_cluster.cc:1428] Setting key bcf65a32802d77962e3a585d7fa20ed2
I20260810 20:59:58.187536 21143 catalog_manager.cc:1347] Generated new cluster ID: 983dc2b0da90484396a90ec9c50a5196
I20260810 20:59:58.187587 21143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:58.202384 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:58.187695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.14.200.62:43075)
0810 20:59:58.187893 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:58.187901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:58.197041 (+ 9140us) server_negotiation.cc:407] Connection header received
0810 20:59:58.197198 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:58.197206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:58.197274 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:58.197407 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:58.197668 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.199060 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.200148 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.200787 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.200956 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.201659 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:58.201663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:58.201663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:58.201880 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:58.201924 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:58.201939 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:58.202009 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:58.202074 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 20:59:58.202166 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260810 20:59:58.203433 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 20:59:58.212085 21143 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:58.212725 21143 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:58.219938 21143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d79a47eca484af6a9e8616e1af53eec: Generated new TSK 0
I20260810 20:59:58.220116 21143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 20:59:58.229866 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:58.226338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.14.200.62:43075)
0810 20:59:58.226381 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:58.226384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:58.226491 (+ 107us) server_negotiation.cc:407] Connection header received
0810 20:59:58.226566 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:58.226566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:58.226576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:58.226636 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0810 20:59:58.226805 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.227275 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.228303 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.228497 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.228546 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.228750 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 20:59:58.229672 (+ 922us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 20:59:58.229797 (+ 125us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 20:59:58.229982 21165 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 20:59:58.226220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43075 (local address 127.0.0.1:46402)
0810 20:59:58.226409 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:58.226422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:58.226494 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:58.226656 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:58.226657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:58.226659 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 20:59:58.226733 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:58.226736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.227360 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.227361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:58.228193 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:58.228196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.228577 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.228578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:58.228644 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.228651 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 20:59:58.229773 (+ 1122us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 20:59:58.229881 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.200.62:43075: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41351/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":112,"threads_started":1}
I20260810 20:59:58.230218 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21056
E20260810 20:59:58.243466 21166 webserver.cc:533] Webserver: sslize failed:
E20260810 20:59:58.243561 21166 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T20:59:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T20:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:58Z Disabled control of system clock
I20260810 20:59:58.279688 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41945
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41945
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/testchainca.cert with env {}
W20260810 20:59:58.357985 21171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:58.358174 21171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:58.358196 21171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:58.359266 21171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:58.359297 21171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:58.359309 21171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:58.359321 21171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:58.359333 21171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 20:59:58.359345 21171 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 20:59:58.359356 21171 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 20:59:58.360730 21171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:58.361001 21171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:58.361177 21171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:58.365758 21177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:58.365768 21178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:58.365756 21180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:58.366035 21171 server_base.cc:1034] running on GCE node
I20260810 20:59:58.366381 21171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:58.366655 21171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:58.367789 21171 hybrid_clock.cc:648] HybridClock initialized: now 1786395598367779 us; error 27 us; skew 500 ppm
I20260810 20:59:58.369040 21171 webserver.cc:460] Webserver started at http://127.14.200.62:35887/ using document root <none> and password file <none>
I20260810 20:59:58.369256 21171 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:58.369297 21171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:58.369469 21171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:58.370483 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "8ef5ef301ca44f15b68e26ba35e01943"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "824240ae3b6e92fb6a285129a5b6d7c5"
server_key_iv: "3f6987b64ff6c018e6e104913861ec96"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.370859 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "8ef5ef301ca44f15b68e26ba35e01943"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "824240ae3b6e92fb6a285129a5b6d7c5"
server_key_iv: "3f6987b64ff6c018e6e104913861ec96"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.372174 21171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 20:59:58.372869 21186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:58.373039 21171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:58.373116 21171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "8ef5ef301ca44f15b68e26ba35e01943"
format_stamp: "Formatted at 2026-08-10 20:59:58 on dist-test-slave-1x4g"
server_key: "824240ae3b6e92fb6a285129a5b6d7c5"
server_key_iv: "3f6987b64ff6c018e6e104913861ec96"
server_key_version: "encryptionkey@0"
I20260810 20:59:58.373164 21171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:58.381345 21171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:58.381552 21171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:58.381640 21171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 20:59:58.385373 21171 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41945
I20260810 20:59:58.385404 21241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41945 every 8 connection(s)
I20260810 20:59:58.385746 21171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 20:59:58.386620 21242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 20:59:58.388710 21242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943: Bootstrap starting.
I20260810 20:59:58.389313 21242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943: Neither blocks nor log segments found. Creating new log.
I20260810 20:59:58.389636 21242 log.cc:824] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943: Log is configured to *not* fsync() on all Append() calls
I20260810 20:59:58.390273 21242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943: No bootstrap required, opened a new log
I20260810 20:59:58.391837 21242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } }
I20260810 20:59:58.391973 21242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 20:59:58.392022 21242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef5ef301ca44f15b68e26ba35e01943, State: Initialized, Role: FOLLOWER
I20260810 20:59:58.392179 21242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } }
I20260810 20:59:58.392251 21242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 20:59:58.392287 21242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 20:59:58.392330 21242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 0 FOLLOWER]: Advancing to term 1
I20260810 20:59:58.392864 21242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } }
I20260810 20:59:58.392963 21242 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef5ef301ca44f15b68e26ba35e01943; no voters:
I20260810 20:59:58.393100 21242 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 20:59:58.393134 21245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 20:59:58.393285 21245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef5ef301ca44f15b68e26ba35e01943, State: Running, Role: LEADER
I20260810 20:59:58.393347 21242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 20:59:58.393380 21245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } }
I20260810 20:59:58.393721 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21171
I20260810 20:59:58.393846 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 20:59:58.394061 15136 external_mini_cluster.cc:1428] Setting key a8686a841144b8d140027b038f9cfdef
I20260810 20:59:58.394529 21246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef5ef301ca44f15b68e26ba35e01943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } } }
I20260810 20:59:58.394622 21246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:58.394843 21247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef5ef301ca44f15b68e26ba35e01943. Latest consensus state: current_term: 1 leader_uuid: "8ef5ef301ca44f15b68e26ba35e01943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef5ef301ca44f15b68e26ba35e01943" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41945 } } }
I20260810 20:59:58.394929 21263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 20:59:58.395120 21247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943 [sys.catalog]: This master's current role is: LEADER
I20260810 20:59:58.395524 21263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 20:59:58.397183 21263 catalog_manager.cc:1347] Generated new cluster ID: 624f176316794f718de32cca673bb7de
I20260810 20:59:58.397235 21263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 20:59:58.400161 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:58.394711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.14.200.62:41945)
0810 20:59:58.395234 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:58.395242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:58.395287 (+ 45us) server_negotiation.cc:407] Connection header received
0810 20:59:58.395399 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:58.395405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:58.395494 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:58.395689 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 20:59:58.395922 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.397259 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.398324 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.398890 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.399012 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.399557 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 20:59:58.399561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 20:59:58.399561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 20:59:58.399735 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0810 20:59:58.399771 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 20:59:58.399783 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 20:59:58.399844 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 20:59:58.399902 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 20:59:58.399976 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":68,"threads_started":1}
I20260810 20:59:58.400928 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 20:59:58.411932 21263 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 20:59:58.412436 21263 catalog_manager.cc:1504] Loading token signing keys...
I20260810 20:59:58.420588 21263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef5ef301ca44f15b68e26ba35e01943: Generated new TSK 0
I20260810 20:59:58.420748 21263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 20:59:58.508628 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 20:59:58.429356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.14.200.62:41945)
0810 20:59:58.429423 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 20:59:58.429425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 20:59:58.429544 (+ 119us) server_negotiation.cc:407] Connection header received
0810 20:59:58.429615 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 20:59:58.429616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 20:59:58.429626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 20:59:58.429682 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0810 20:59:58.429832 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.430320 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.431445 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.431615 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.431658 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.431858 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 20:59:58.508403 (+ 76545us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 20:59:58.508572 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36790: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 20:59:58.508714 21264 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36790: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 20:59:58.508852 21273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 20:59:58.429305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41945 (local address 127.0.0.1:36790)
0810 20:59:58.429474 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 20:59:58.429486 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 20:59:58.429550 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 20:59:58.429707 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 20:59:58.429708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 20:59:58.429710 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 20:59:58.429769 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:58.429771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.430400 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.430401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:58.431359 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 20:59:58.431362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 20:59:58.431695 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 20:59:58.431696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 20:59:58.431763 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 20:59:58.431769 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 20:59:58.508565 (+ 76796us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 20:59:58.508723 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.200.62:41945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260810 20:59:58.509234 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21171
2026-08-10T20:59:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (258 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 20:59:58 dist-test-slave-1x4g krb5kdc[21285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 20:59:58 dist-test-slave-1x4g krb5kdc[21285](info): set up 2 sockets
Aug 10 20:59:58 dist-test-slave-1x4g krb5kdc[21285](info): commencing operation
krb5kdc: starting...
W20260810 20:59:59.769050 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 20:59:59 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T20:59:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T20:59:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 20:59:59.886953 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42565
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:42565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 20:59:59.967082 21301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 20:59:59.967273 21301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 20:59:59.967293 21301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 20:59:59.968411 21301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 20:59:59.968452 21301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 20:59:59.968466 21301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 20:59:59.968478 21301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 20:59:59.968528 21301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 20:59:59.969889 21301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:42565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 20:59:59.970124 21301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 20:59:59.970327 21301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 20:59:59.974974 21310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:59.974953 21306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 20:59:59.974959 21308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 20:59:59.975112 21301 server_base.cc:1034] running on GCE node
I20260810 20:59:59.975422 21301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 20:59:59.975703 21301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 20:59:59.976846 21301 hybrid_clock.cc:648] HybridClock initialized: now 1786395599976825 us; error 30 us; skew 500 ppm
Aug 10 20:59:59 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395599, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 20:59:59.979307 21301 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 20:59:59.979736 21301 webserver.cc:460] Webserver started at http://127.14.200.62:41159/ using document root <none> and password file <none>
I20260810 20:59:59.979993 21301 fs_manager.cc:359] Metadata directory not provided
I20260810 20:59:59.980062 21301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 20:59:59.980165 21301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 20:59:59.981243 21301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "58d9e7876f2d4e659bd58412cf5cbcdf"
format_stamp: "Formatted at 2026-08-10 20:59:59 on dist-test-slave-1x4g"
server_key: "2ae1e6fcdfe45a7789c0c9d92065c404"
server_key_iv: "0d074f80f03c41a205879dacfc1ce7ef"
server_key_version: "encryptionkey@0"
I20260810 20:59:59.981560 21301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "58d9e7876f2d4e659bd58412cf5cbcdf"
format_stamp: "Formatted at 2026-08-10 20:59:59 on dist-test-slave-1x4g"
server_key: "2ae1e6fcdfe45a7789c0c9d92065c404"
server_key_iv: "0d074f80f03c41a205879dacfc1ce7ef"
server_key_version: "encryptionkey@0"
I20260810 20:59:59.982769 21301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 20:59:59.983412 21319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 20:59:59.983584 21301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 20:59:59.983681 21301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "58d9e7876f2d4e659bd58412cf5cbcdf"
format_stamp: "Formatted at 2026-08-10 20:59:59 on dist-test-slave-1x4g"
server_key: "2ae1e6fcdfe45a7789c0c9d92065c404"
server_key_iv: "0d074f80f03c41a205879dacfc1ce7ef"
server_key_version: "encryptionkey@0"
I20260810 20:59:59.983762 21301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 20:59:59.996665 21301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 20:59:59.998382 21301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 20:59:59.998538 21301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:00.002982 21301 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:42565
I20260810 21:00:00.003024 21378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:42565 every 8 connection(s)
I20260810 21:00:00.003384 21301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:00.004244 21379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:00.006680 21379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf: Bootstrap starting.
I20260810 21:00:00.007290 21379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:00.007601 21379 log.cc:824] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:00.008337 21379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf: No bootstrap required, opened a new log
I20260810 21:00:00.010107 21379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } }
I20260810 21:00:00.010267 21379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:00.010330 21379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d9e7876f2d4e659bd58412cf5cbcdf, State: Initialized, Role: FOLLOWER
I20260810 21:00:00.010524 21379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } }
I20260810 21:00:00.010622 21379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:00.010681 21379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:00.010742 21379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:00.011293 21379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } }
I20260810 21:00:00.011422 21379 leader_election.cc:302] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58d9e7876f2d4e659bd58412cf5cbcdf; no voters:
I20260810 21:00:00.011623 21379 leader_election.cc:288] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:00.011689 21383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:00.011704 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21301
I20260810 21:00:00.011799 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:00.011914 21379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:00.012064 15136 external_mini_cluster.cc:1428] Setting key 00cbccd6f5ce705da3eae3f30a4fee2e
I20260810 21:00:00.012099 21383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [term 1 LEADER]: Becoming Leader. State: Replica: 58d9e7876f2d4e659bd58412cf5cbcdf, State: Running, Role: LEADER
I20260810 21:00:00.012192 21383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } }
I20260810 21:00:00.012639 21384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } } }
I20260810 21:00:00.012742 21384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:00.012934 21385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58d9e7876f2d4e659bd58412cf5cbcdf. Latest consensus state: current_term: 1 leader_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d9e7876f2d4e659bd58412cf5cbcdf" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42565 } } }
I20260810 21:00:00.013008 21385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:00.013254 21395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:00.013942 21395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:00.015612 21395 catalog_manager.cc:1347] Generated new cluster ID: aead141481bb4514b1929dab88269859
I20260810 21:00:00.015674 21395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:00.024320 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.012731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.14.200.62:42565)
0810 21:00:00.012944 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.012954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.012984 (+ 30us) server_negotiation.cc:407] Connection header received
0810 21:00:00.013098 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.013105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.013162 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.014118 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.014224 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.015522 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.016689 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.017311 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.017427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.020431 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.020436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.020436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.020448 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.022602 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.023012 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.023015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.023016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.023131 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.023348 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.023349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.023349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.023687 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.023844 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.023948 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.024092 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
I20260810 21:00:00.027355 21395 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:00.027944 21395 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:00.032698 21395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58d9e7876f2d4e659bd58412cf5cbcdf: Generated new TSK 0
I20260810 21:00:00.032902 21395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:00:00.067189 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:00.168758 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:00.168995 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:00.169018 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:00.170092 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:00.170130 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:00.170152 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:00.171581 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:00.171808 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:00.172037 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:00.177138 21413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:00.177222 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:00.177119 21414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:00.177336 21408 server_base.cc:1034] running on GCE node
I20260810 21:00:00.177631 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:00.177906 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:00.179096 21408 hybrid_clock.cc:648] HybridClock initialized: now 1786395600179070 us; error 40 us; skew 500 ppm
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.181967 21408 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:00:00.182366 21408 webserver.cc:460] Webserver started at http://127.14.200.1:46445/ using document root <none> and password file <none>
I20260810 21:00:00.182585 21408 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:00.182629 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:00.182772 21408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:00.183799 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "37049f94489644ab96bf0e119c8ed9e2"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "fb970391c38c4b8d36ae37c7ed0efc01"
server_key_iv: "abe8d524cd61f6a2e5a5c281c944758b"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.184165 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "37049f94489644ab96bf0e119c8ed9e2"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "fb970391c38c4b8d36ae37c7ed0efc01"
server_key_iv: "abe8d524cd61f6a2e5a5c281c944758b"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.185573 21408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:00.186369 21425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.186553 21408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:00.186625 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "37049f94489644ab96bf0e119c8ed9e2"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "fb970391c38c4b8d36ae37c7ed0efc01"
server_key_iv: "abe8d524cd61f6a2e5a5c281c944758b"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.186705 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:00.211695 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:00.213784 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:00.213932 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:00.214184 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:00.214497 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:00.214530 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.214605 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:00.214630 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.220242 21408 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46307
I20260810 21:00:00.220340 21557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46307 every 8 connection(s)
I20260810 21:00:00.220633 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:00.223487 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21408
I20260810 21:00:00.223579 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:00.223769 15136 external_mini_cluster.cc:1428] Setting key d1bd29bbe9a661a71c841dedc724d62b
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:00.231174 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.221407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53227 (local address 127.14.200.62:42565)
0810 21:00:00.221451 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.221452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.221675 (+ 223us) server_negotiation.cc:407] Connection header received
0810 21:00:00.222678 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.222679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.222698 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.222748 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.223974 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.224691 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.226002 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.226230 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.226323 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.228320 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.228321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.228321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.228329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.229731 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.230158 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.230160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.230160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.230225 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.230479 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.230480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.230480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.230604 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.230677 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.230991 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.231083 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:00:00.231364 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.221370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42565 (local address 127.14.200.1:53227)
0810 21:00:00.221561 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:00.221596 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:00.222556 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:00.222815 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:00.222819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:00.223078 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:00.223257 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.223262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.224821 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.224822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.225900 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.225903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.226336 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.226337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.226457 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.226943 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:00.226951 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:00.228225 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:00.229834 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.229842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.229844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.230080 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.230297 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.230297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.230298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.230407 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.230670 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:00.230677 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:00.230821 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 21:00:00.230942 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 21:00:00.231124 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260810 21:00:00.231911 21558 heartbeater.cc:347] Connected to a master server at 127.14.200.62:42565
I20260810 21:00:00.231997 21558 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:00.232201 21558 heartbeater.cc:511] Master 127.14.200.62:42565 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
I20260810 21:00:00.232828 21338 ts_manager.cc:194] Registered new tserver with Master: 37049f94489644ab96bf0e119c8ed9e2 (127.14.200.1:46307)
I20260810 21:00:00.233412 21338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:53227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:00:00.249621 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:00.330724 21563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:00.330935 21563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:00.330960 21563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:00.332055 21563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:00.332087 21563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:00.332113 21563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:00.333587 21563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:00.333782 21563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:00.333972 21563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:00.339030 21572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:00.339001 21570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:00.339052 21563 server_base.cc:1034] running on GCE node
W20260810 21:00:00.339004 21569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:00.339355 21563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:00.339599 21563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:00.340763 21563 hybrid_clock.cc:648] HybridClock initialized: now 1786395600340750 us; error 47 us; skew 500 ppm
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.343299 21563 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:00:00.343719 21563 webserver.cc:460] Webserver started at http://127.14.200.2:46091/ using document root <none> and password file <none>
I20260810 21:00:00.343942 21563 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:00.344007 21563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:00.344159 21563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:00.345369 21563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "588c90589eca47e09d64310f86de4372"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "60f2c22a27c66ad47c1183b2cc510370"
server_key_iv: "dd817f78cd3b06a8103e32f4ac74514d"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.345764 21563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "588c90589eca47e09d64310f86de4372"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "60f2c22a27c66ad47c1183b2cc510370"
server_key_iv: "dd817f78cd3b06a8103e32f4ac74514d"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.347065 21563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:00.347767 21579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.347955 21563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:00.348052 21563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "588c90589eca47e09d64310f86de4372"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "60f2c22a27c66ad47c1183b2cc510370"
server_key_iv: "dd817f78cd3b06a8103e32f4ac74514d"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.348141 21563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:00.355643 21563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:00.357957 21563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:00.358143 21563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:00.358369 21563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:00.358675 21563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:00.358726 21563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.358767 21563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:00.358793 21563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.364365 21563 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:39583
I20260810 21:00:00.364439 21704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:39583 every 8 connection(s)
I20260810 21:00:00.364811 21563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:00.373310 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.365587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60189 (local address 127.14.200.62:42565)
0810 21:00:00.365643 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.365645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.365829 (+ 184us) server_negotiation.cc:407] Connection header received
0810 21:00:00.366753 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.366754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.366763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.366811 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.367385 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.367866 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.368907 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.369097 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.369150 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.371005 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.371007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.371007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.371019 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.372136 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.372507 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.372508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.372508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.372559 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.372812 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.372812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.372813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.372918 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.372967 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.373188 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.373250 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:00:00.373507 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.365559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42565 (local address 127.14.200.2:60189)
0810 21:00:00.365708 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:00.365743 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:00.366666 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:00.366881 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:00.366884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:00.367123 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:00.367314 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.367319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.367967 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.367968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.368746 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.368748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.369169 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.369170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.369272 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.369676 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:00.369684 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:00.370924 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:00.372205 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.372211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.372213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.372445 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.372615 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.372615 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.372616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.372710 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.372972 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:00.372977 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:00.373082 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 21:00:00.373161 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 21:00:00.373274 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260810 21:00:00.373978 21705 heartbeater.cc:347] Connected to a master server at 127.14.200.62:42565
I20260810 21:00:00.374183 21705 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:00.374382 21705 heartbeater.cc:511] Master 127.14.200.62:42565 requested a full tablet report, sending...
I20260810 21:00:00.374790 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21563
I20260810 21:00:00.374845 21338 ts_manager.cc:194] Registered new tserver with Master: 588c90589eca47e09d64310f86de4372 (127.14.200.2:39583)
I20260810 21:00:00.374898 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:00.375098 15136 external_mini_cluster.cc:1428] Setting key 4ad8e8000dec40fe563ba998e67b295a
I20260810 21:00:00.375322 21338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:60189
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:00:00.403162 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:00.484023 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:00.484225 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:00.484248 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:00.485435 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:00.485474 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:00.485502 21711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:00.487046 21711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:00.487272 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:00.487453 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:00.492388 21716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:00.492445 21719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:00.492385 21717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:00.492638 21711 server_base.cc:1034] running on GCE node
I20260810 21:00:00.492883 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:00.493160 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:00.494306 21711 hybrid_clock.cc:648] HybridClock initialized: now 1786395600494296 us; error 33 us; skew 500 ppm
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.496919 21711 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:00:00.497323 21711 webserver.cc:460] Webserver started at http://127.14.200.3:43821/ using document root <none> and password file <none>
I20260810 21:00:00.497519 21711 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:00.497560 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:00.497651 21711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:00.498737 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "59743a80d35a483880ff5b3e8585585b"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "0477ddf6441d3a4ed47ce07a0d553df7"
server_key_iv: "a72d68823a4a59c3be918b264e713834"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.499046 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "59743a80d35a483880ff5b3e8585585b"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "0477ddf6441d3a4ed47ce07a0d553df7"
server_key_iv: "a72d68823a4a59c3be918b264e713834"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.500531 21711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:00:00.501283 21726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.501446 21711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.501555 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "59743a80d35a483880ff5b3e8585585b"
format_stamp: "Formatted at 2026-08-10 21:00:00 on dist-test-slave-1x4g"
server_key: "0477ddf6441d3a4ed47ce07a0d553df7"
server_key_iv: "a72d68823a4a59c3be918b264e713834"
server_key_version: "encryptionkey@0"
I20260810 21:00:00.501631 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:00.517143 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:00.518985 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:00.519142 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:00.519378 21711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:00.519718 21711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:00.519771 21711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.519814 21711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:00.519840 21711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:00.525781 21711 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:33283
I20260810 21:00:00.525825 21853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:33283 every 8 connection(s)
I20260810 21:00:00.526192 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:00.528802 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21711
I20260810 21:00:00.528882 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:00.529080 15136 external_mini_cluster.cc:1428] Setting key 2e5df7dc6e371064fe56ca50277f17dd
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:00.535784 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.527004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:50877 (local address 127.14.200.62:42565)
0810 21:00:00.527060 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.527063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.527290 (+ 227us) server_negotiation.cc:407] Connection header received
0810 21:00:00.528220 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.528221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.528231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.528281 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.528878 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.529618 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.530665 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.530845 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.530926 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.533153 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.533154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.533154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.533162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.534316 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.534779 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.534781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.534781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.534841 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.535109 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.535110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.535110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.535236 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.535308 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.535607 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.535689 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 21:00:00.536003 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.526987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42565 (local address 127.14.200.3:50877)
0810 21:00:00.527159 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:00.527210 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:00.528127 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:00.528331 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:00.528334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:00.528569 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:00.528753 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.528756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.529713 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.529715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.530591 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:00.530594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.530957 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.530958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:00.531054 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.531603 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:00.531611 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:00.533056 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:00.534415 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.534422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.534424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.534682 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.534918 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:00.534919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:00.534920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:00.535039 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:00.535312 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:00.535319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:00.535467 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 21:00:00.535583 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 21:00:00.535746 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260810 21:00:00.536478 21854 heartbeater.cc:347] Connected to a master server at 127.14.200.62:42565
I20260810 21:00:00.536587 21854 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:00.536871 21854 heartbeater.cc:511] Master 127.14.200.62:42565 requested a full tablet report, sending...
I20260810 21:00:00.537264 21338 ts_manager.cc:194] Registered new tserver with Master: 59743a80d35a483880ff5b3e8585585b (127.14.200.3:33283)
I20260810 21:00:00.537653 21338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:50877
I20260810 21:00:00.540558 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/test-admin.keytab.
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.558297 15136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 21:00:00.560063 21861 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:00.559203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.14.200.1:46307)
0810 21:00:00.559375 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.559380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.559390 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:00.559428 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.559430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.559443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.559516 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.559949 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49586: BlockingRecv error: recv got EOF from 127.0.0.1:49586 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
W20260810 21:00:00.560141 15136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc' of type FILE
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.561127 15136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.561982 15136 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 21:00:00.569445 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.562499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49602 (local address 127.14.200.1:46307)
0810 21:00:00.562544 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.562546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.562595 (+ 49us) server_negotiation.cc:407] Connection header received
0810 21:00:00.562634 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.562634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.562644 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.562689 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.563055 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.563774 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.564450 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.564842 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.564909 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.567039 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.567047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.567048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.567054 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.568391 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.568727 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.568729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.568729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.568841 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.569010 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.569010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.569011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.569145 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.569258 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.569317 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.569375 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260810 21:00:00.571029 21861 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:00.570604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.14.200.1:46307)
0810 21:00:00.570636 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.570638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.570665 (+ 27us) server_negotiation.cc:407] Connection header received
0810 21:00:00.570707 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.570708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.570716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.570763 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.570974 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49612: BlockingRecv error: recv got EOF from 127.0.0.1:49612 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260810 21:00:00.571201 15136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc' of type FILE
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.572155 15136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.573033 15136 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 21:00:00.579500 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.573500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.14.200.1:46307)
0810 21:00:00.573552 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.573553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.573595 (+ 42us) server_negotiation.cc:407] Connection header received
0810 21:00:00.573674 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.573675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.573686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.573758 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.574066 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.574491 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.575147 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.575338 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.575392 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.577279 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.577280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.577280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.577286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.578666 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.578956 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.578956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.578957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.579005 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.579142 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.579142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.579142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.579218 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.579277 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.579376 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.579428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 21:00:00.581241 21861 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:00.580545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.14.200.1:46307)
0810 21:00:00.580644 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.580645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.580680 (+ 35us) server_negotiation.cc:407] Connection header received
0810 21:00:00.580743 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.580744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.580751 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.580851 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.581202 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49628: BlockingRecv error: recv got EOF from 127.0.0.1:49628 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 21:00:00.581323 15136 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786395574307397-15136-0/krb5kdc/krb5cc' of type FILE
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.582257 15136 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:00.583094 15136 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.1@KRBTEST.COM
I20260810 21:00:00.589651 21861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:00.583578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.14.200.1:46307)
0810 21:00:00.583637 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:00.583639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:00.583713 (+ 74us) server_negotiation.cc:407] Connection header received
0810 21:00:00.583793 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:00.583794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:00.583802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:00.583869 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:00.584224 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.584687 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.585391 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:00.585587 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:00.585690 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:00.587551 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:00.587552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:00.587552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:00.587558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:00.588726 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.589087 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.589088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.589088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.589132 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:00.589286 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:00.589286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:00.589287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:00.589366 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:00.589467 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:00.589537 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 21:00:00.589601 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:00:00.590298 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21408
I20260810 21:00:00.597919 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21563
I20260810 21:00:00.603302 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21711
I20260810 21:00:00.611563 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21301
2026-08-10T21:00:00Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1926 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21874](info): set up 2 sockets
Aug 10 21:00:00 dist-test-slave-1x4g krb5kdc[21874](info): commencing operation
krb5kdc: starting...
W20260810 21:00:01.668838 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:01 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab.
Entry for principal oryx/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab.
I20260810 21:00:01.801075 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39009
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab
--principal=oryx/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:01.885143 21888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:01.885387 21888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:01.885426 21888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:01.886570 21888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:01.886646 21888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:01.886670 21888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:01.886693 21888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:01.886720 21888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:01.888249 21888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab
--principal=oryx/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:01.888525 21888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:01.888788 21888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:01.893764 21897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:01.893786 21894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:01.893786 21895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:01.894085 21888 server_base.cc:1034] running on GCE node
I20260810 21:00:01.894351 21888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:01.894662 21888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:01.895809 21888 hybrid_clock.cc:648] HybridClock initialized: now 1786395601895787 us; error 37 us; skew 500 ppm
Aug 10 21:00:01 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:01.904340 21888 init.cc:377] Logged in from keytab as oryx/127.14.200.62@KRBTEST.COM (short username oryx)
I20260810 21:00:01.904938 21888 webserver.cc:460] Webserver started at http://127.14.200.62:41337/ using document root <none> and password file <none>
I20260810 21:00:01.905220 21888 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:01.905268 21888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:01.905454 21888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:01.906535 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "c31bf9e284f94a21866c0aaa6b293f14"
format_stamp: "Formatted at 2026-08-10 21:00:01 on dist-test-slave-1x4g"
server_key: "079bc2b54c2bc06294ce1f86f5a3c95b"
server_key_iv: "b2b02243fc16add56770b59a61046ef4"
server_key_version: "encryptionkey@0"
I20260810 21:00:01.906859 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "c31bf9e284f94a21866c0aaa6b293f14"
format_stamp: "Formatted at 2026-08-10 21:00:01 on dist-test-slave-1x4g"
server_key: "079bc2b54c2bc06294ce1f86f5a3c95b"
server_key_iv: "b2b02243fc16add56770b59a61046ef4"
server_key_version: "encryptionkey@0"
I20260810 21:00:01.908289 21888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:01.909039 21904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:01.909231 21888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:01.909312 21888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "c31bf9e284f94a21866c0aaa6b293f14"
format_stamp: "Formatted at 2026-08-10 21:00:01 on dist-test-slave-1x4g"
server_key: "079bc2b54c2bc06294ce1f86f5a3c95b"
server_key_iv: "b2b02243fc16add56770b59a61046ef4"
server_key_version: "encryptionkey@0"
I20260810 21:00:01.909368 21888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:01.932088 21888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:01.934234 21888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:01.934352 21888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:01.938865 21888 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:39009
I20260810 21:00:01.938956 21963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:39009 every 8 connection(s)
I20260810 21:00:01.939311 21888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:01.940338 21964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:01.942811 21964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Bootstrap starting.
I20260810 21:00:01.943428 21964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:01.943755 21964 log.cc:824] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:01.944499 21964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: No bootstrap required, opened a new log
I20260810 21:00:01.946147 21964 raft_consensus.cc:348] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } }
I20260810 21:00:01.946291 21964 raft_consensus.cc:374] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:01.946313 21964 raft_consensus.cc:729] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c31bf9e284f94a21866c0aaa6b293f14, State: Initialized, Role: FOLLOWER
I20260810 21:00:01.946503 21964 consensus_queue.cc:260] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } }
I20260810 21:00:01.946570 21964 raft_consensus.cc:388] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:01.946612 21964 raft_consensus.cc:482] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:01.946664 21964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:01.947252 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21888
I20260810 21:00:01.947389 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:01.947602 15136 external_mini_cluster.cc:1428] Setting key 2db1e89f6601ea48bee435acdf89e371
I20260810 21:00:01.947770 21964 raft_consensus.cc:504] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } }
I20260810 21:00:01.947881 21964 leader_election.cc:302] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c31bf9e284f94a21866c0aaa6b293f14; no voters:
I20260810 21:00:01.948087 21964 leader_election.cc:288] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:01.948110 21967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:01.948342 21967 raft_consensus.cc:686] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 LEADER]: Becoming Leader. State: Replica: c31bf9e284f94a21866c0aaa6b293f14, State: Running, Role: LEADER
I20260810 21:00:01.948458 21964 sys_catalog.cc:564] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:01.948474 21967 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } }
I20260810 21:00:01.948812 21967 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } }
I20260810 21:00:01.948977 21967 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:01.949184 21967 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c31bf9e284f94a21866c0aaa6b293f14. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } }
I20260810 21:00:01.949307 21967 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:01.949656 21977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:01.950320 21977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:01.952826 21977 catalog_manager.cc:1347] Generated new cluster ID: 87e6f592ab354a309a2ebb8bb9f0c966
I20260810 21:00:01.952881 21977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:00:01 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:01.960071 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:01.948215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.14.200.62:39009)
0810 21:00:01.948629 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:01.948640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:01.948681 (+ 41us) server_negotiation.cc:407] Connection header received
0810 21:00:01.948866 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:01.948874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:01.948938 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:01.950113 (+ 1175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:01.950221 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:01.951404 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:01.952238 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:01.952886 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:01.953042 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:01.955603 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:01.955608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:01.955608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:01.955629 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:01.958006 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:01.958439 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:01.958443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:01.958443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:01.958562 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:01.958799 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:01.958801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:01.958801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:01.959359 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:01.959577 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:01.959683 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 21:00:01.959832 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":107,"threads_started":1}
I20260810 21:00:01.974676 21977 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:01.975256 21977 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:01.978930 21977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Generated new TSK 0
I20260810 21:00:01.979148 21977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
I20260810 21:00:02.007517 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:02.091737 21991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:02.092000 21991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:02.092041 21991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:02.093539 21991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:02.093596 21991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:02.093637 21991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:02.095268 21991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:02.095566 21991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:02.095831 21991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.100867 21997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.100881 21996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.100890 21999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.101311 21991 server_base.cc:1034] running on GCE node
I20260810 21:00:02.101547 21991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.101768 21991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.102936 21991 hybrid_clock.cc:648] HybridClock initialized: now 1786395602102919 us; error 27 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.105746 21991 init.cc:377] Logged in from keytab as oryx/127.14.200.1@KRBTEST.COM (short username oryx)
I20260810 21:00:02.106256 21991 webserver.cc:460] Webserver started at http://127.14.200.1:34697/ using document root <none> and password file <none>
I20260810 21:00:02.106535 21991 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.106586 21991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.106778 21991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:02.107964 21991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "d9ee17ce95c946ebbce389522fe34c40"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "a9c0cf458e15cee5d534224207951e3c"
server_key_iv: "0a9b7c2bf7562fe25a8827b4bf75fd6e"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.108368 21991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "d9ee17ce95c946ebbce389522fe34c40"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "a9c0cf458e15cee5d534224207951e3c"
server_key_iv: "0a9b7c2bf7562fe25a8827b4bf75fd6e"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.109937 21991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260810 21:00:02.110713 22008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.110886 21991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:02.111011 21991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "d9ee17ce95c946ebbce389522fe34c40"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "a9c0cf458e15cee5d534224207951e3c"
server_key_iv: "0a9b7c2bf7562fe25a8827b4bf75fd6e"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.111101 21991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.122712 21991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.124760 21991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:02.125020 21991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.125295 21991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:02.125662 21991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:02.125725 21991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.125766 21991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:02.125820 21991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.132151 21991 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42973
I20260810 21:00:02.132236 22130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42973 every 8 connection(s)
I20260810 21:00:02.132622 21991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:02.133713 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 21991
I20260810 21:00:02.133828 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:02.134084 15136 external_mini_cluster.cc:1428] Setting key 83eae56fa43fe4cfff1e08682dbf3416
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.14.200.2@KRBTEST.COM; defaulting to no policy
I20260810 21:00:02.143733 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.133499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:41157 (local address 127.14.200.62:39009)
0810 21:00:02.133584 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.133587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.133979 (+ 392us) server_negotiation.cc:407] Connection header received
0810 21:00:02.134661 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.134663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.134678 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.134749 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.136309 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.136915 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.137965 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.138194 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.138275 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.140601 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.140602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.140603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.140613 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.142172 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.142641 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.142643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.142643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.142705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.142975 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.142976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.142977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.143111 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.143194 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.143504 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.143620 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 21:00:02.143924 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.133433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:41157)
0810 21:00:02.133707 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.133767 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.134549 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.134840 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.134845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.135853 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.136196 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.136204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.137029 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.137030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.137860 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.137863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.138290 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.138291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.138433 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.139026 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.139036 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.140487 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.142281 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.142289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.142292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.142548 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.142782 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.142783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.142783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.142906 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.143196 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.143205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.143362 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.143480 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.143671 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":128,"threads_started":1}
I20260810 21:00:02.144503 22131 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:02.144613 22131 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:02.144883 22131 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:02.145536 21925 ts_manager.cc:194] Registered new tserver with Master: d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:02.146252 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:41157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
I20260810 21:00:02.164654 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:02.249614 22136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:02.249907 22136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:02.249933 22136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:02.251089 22136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:02.251127 22136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:02.251153 22136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:02.252553 22136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:02.252743 22136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:02.253003 22136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.258097 22141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.258112 22142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.258113 22145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.258385 22136 server_base.cc:1034] running on GCE node
I20260810 21:00:02.258632 22136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.258879 22136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.260044 22136 hybrid_clock.cc:648] HybridClock initialized: now 1786395602260030 us; error 26 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.262529 22136 init.cc:377] Logged in from keytab as oryx/127.14.200.2@KRBTEST.COM (short username oryx)
I20260810 21:00:02.263017 22136 webserver.cc:460] Webserver started at http://127.14.200.2:38567/ using document root <none> and password file <none>
I20260810 21:00:02.263243 22136 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.263309 22136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.263468 22136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:02.264531 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "08ffe0f6df39453ea94a10b8630e582c"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "4c7fbea38cf67089216d27c33adbb6d6"
server_key_iv: "d40446f2700dd1ada97b7f49548b1f34"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.264938 22136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "08ffe0f6df39453ea94a10b8630e582c"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "4c7fbea38cf67089216d27c33adbb6d6"
server_key_iv: "d40446f2700dd1ada97b7f49548b1f34"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.266276 22136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:02.266968 22153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.267149 22136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:02.267254 22136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "08ffe0f6df39453ea94a10b8630e582c"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "4c7fbea38cf67089216d27c33adbb6d6"
server_key_iv: "d40446f2700dd1ada97b7f49548b1f34"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.267329 22136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.296888 22136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.298871 22136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:02.299039 22136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.299291 22136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:02.299625 22136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:02.299680 22136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.299726 22136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:02.299758 22136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.305693 22136 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:43259
I20260810 21:00:02.305742 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:43259 every 8 connection(s)
I20260810 21:00:02.306067 22136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:00:02.310949 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22136
I20260810 21:00:02.311057 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:02.311264 15136 external_mini_cluster.cc:1428] Setting key 66559489a6dc5aa30b470de910f19cfc
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:02.315828 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.306777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:43379)
0810 21:00:02.306956 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.306993 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.307476 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.307671 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.307674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.308271 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.308478 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.308482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.309154 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.309155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.309948 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.309950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.310303 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.310304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.310403 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.310900 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.310911 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.312681 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.314277 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.314284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.314287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.314643 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.314859 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.314860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.314861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.315013 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.315246 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.315253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.315426 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.315539 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.315666 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
I20260810 21:00:02.315826 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.306801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:43379 (local address 127.14.200.62:39009)
0810 21:00:02.306868 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.306871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.307080 (+ 209us) server_negotiation.cc:407] Connection header received
0810 21:00:02.307556 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.307557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.307568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.307620 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.308552 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.309077 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.310033 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.310229 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.310282 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.312796 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.312798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.312798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.312805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.314169 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.314736 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.314737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.314737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.314790 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.315080 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.315080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.315081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.315182 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.315232 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.315584 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.315684 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 21:00:02.316360 22278 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:02.316486 22278 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:02.316677 22278 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:02.317147 21925 ts_manager.cc:194] Registered new tserver with Master: 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259)
I20260810 21:00:02.317705 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:43379
WARNING: no policy specified for oryx/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
I20260810 21:00:02.339213 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:02.423723 22284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:02.423976 22284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:02.424016 22284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:02.425333 22284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:02.425395 22284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:02.425441 22284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:02.426923 22284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:02.427186 22284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:02.427418 22284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.432511 22289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.432575 22284 server_base.cc:1034] running on GCE node
W20260810 21:00:02.432534 22290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.432523 22292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.433099 22284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.433349 22284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.434530 22284 hybrid_clock.cc:648] HybridClock initialized: now 1786395602434501 us; error 41 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.437044 22284 init.cc:377] Logged in from keytab as oryx/127.14.200.3@KRBTEST.COM (short username oryx)
I20260810 21:00:02.437498 22284 webserver.cc:460] Webserver started at http://127.14.200.3:35119/ using document root <none> and password file <none>
I20260810 21:00:02.437723 22284 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.437779 22284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.437870 22284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:02.438866 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "89564e8aaf4f48f49a2b09b76363d9ca"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "675e548e36f65b7d7e958dcc71c4c264"
server_key_iv: "51b54c83d69e32983443c83206c7465f"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.439170 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "89564e8aaf4f48f49a2b09b76363d9ca"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "675e548e36f65b7d7e958dcc71c4c264"
server_key_iv: "51b54c83d69e32983443c83206c7465f"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.440438 22284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:02.441188 22300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.441382 22284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:00:02.441475 22284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "89564e8aaf4f48f49a2b09b76363d9ca"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "675e548e36f65b7d7e958dcc71c4c264"
server_key_iv: "51b54c83d69e32983443c83206c7465f"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.441532 22284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.453137 22284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.455422 22284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:02.455533 22284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.455708 22284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:02.455969 22284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:02.456001 22284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.456022 22284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:02.456034 22284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.462018 22284 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:42721
I20260810 21:00:02.462090 22421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:42721 every 8 connection(s)
I20260810 21:00:02.462430 22284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:02.464560 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22284
I20260810 21:00:02.464663 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:02.464891 15136 external_mini_cluster.cc:1428] Setting key 4d747ea41cdc715754bfa7e65beee84e
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:02.471576 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.463209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:52067 (local address 127.14.200.62:39009)
0810 21:00:02.463304 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.463306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.463478 (+ 172us) server_negotiation.cc:407] Connection header received
0810 21:00:02.463965 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.463967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.463978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.464037 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.465116 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.465695 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.466694 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.466887 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.466944 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.468975 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.468976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.468976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.468983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.470242 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.470633 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.470634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.470635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.470700 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.470943 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.470944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.470944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.471061 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.471129 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.471404 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.471483 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 21:00:02.471771 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.463171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:52067)
0810 21:00:02.463331 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.463373 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.463877 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.464090 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.464093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.464695 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.465004 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.465011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.465802 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.465803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.466613 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.466615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.466964 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.466965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.467071 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.467515 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.467523 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.468894 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.470322 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.470329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.470331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.470553 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.470766 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.470767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.470767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.470871 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.471132 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.471139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.471276 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.471380 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.471543 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260810 21:00:02.472210 22423 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:02.472302 22423 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:02.472450 22423 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:02.472846 21925 ts_manager.cc:194] Registered new tserver with Master: 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3:42721)
I20260810 21:00:02.473189 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:52067
I20260810 21:00:02.476544 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.200.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.61.keytab.
Entry for principal oryx/127.14.200.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.61.keytab.
I20260810 21:00:02.503990 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:38445
--webserver_interface=127.14.200.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.61.keytab
--principal=oryx/127.14.200.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39009,127.14.200.61:38445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:02.587777 22431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:02.587980 22431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:02.588003 22431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:02.589188 22431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:02.589228 22431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:02.589242 22431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:02.589255 22431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:02.589267 22431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:02.590739 22431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39009,127.14.200.61:38445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.61.keytab
--principal=oryx/127.14.200.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:38445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.200.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
W20260810 21:00:02.590862 22431 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.200.62:39009,127.14.200.61:38445'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 21:00:02.591008 22431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:02.591243 22431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.595981 22438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.595966 22437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.595963 22440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.596302 22431 server_base.cc:1034] running on GCE node
I20260810 21:00:02.596567 22431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.596915 22431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.598085 22431 hybrid_clock.cc:648] HybridClock initialized: now 1786395602598062 us; error 37 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.600581 22431 init.cc:377] Logged in from keytab as oryx/127.14.200.61@KRBTEST.COM (short username oryx)
I20260810 21:00:02.601109 22431 webserver.cc:460] Webserver started at http://127.14.200.61:41371/ using document root <none> and password file <none>
I20260810 21:00:02.601349 22431 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.601392 22431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.601538 22431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:02.602568 22431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/instance:
uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "fd198296baa68d0059b2b61cc8b2d026"
server_key_iv: "1b4a27becb17a51b9c4d9f05e13e9a38"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.602917 22431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal/instance:
uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "fd198296baa68d0059b2b61cc8b2d026"
server_key_iv: "1b4a27becb17a51b9c4d9f05e13e9a38"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.604314 22431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:00:02.605067 22448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.605239 22431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:02.605341 22431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "fd198296baa68d0059b2b61cc8b2d026"
server_key_iv: "1b4a27becb17a51b9c4d9f05e13e9a38"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.605422 22431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.623911 22431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.625854 22431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:02.626053 22431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.630393 22431 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.61:38445
I20260810 21:00:02.630434 22505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.61:38445 every 8 connection(s)
I20260810 21:00:02.630813 22431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
I20260810 21:00:02.631798 22506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:02.633435 22506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.61@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:02.639897 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22431
I20260810 21:00:02.643064 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.634115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.14.200.62:39009)
0810 21:00:02.634180 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.634182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.634403 (+ 221us) server_negotiation.cc:407] Connection header received
0810 21:00:02.634900 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.634902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.634915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.634964 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.635593 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.636102 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.637204 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.637409 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.637494 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.639655 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.639657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.639657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.639665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.641484 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.641942 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.641944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.641944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.642009 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.642296 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.642297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.642297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.642451 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.642507 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.642884 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.642971 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 21:00:02.643153 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.634089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38038)
0810 21:00:02.634280 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.634314 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.634790 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.635051 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.635055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.635297 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.635487 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.635491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.636206 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.636207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.637102 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.637105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.637504 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.637505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.637628 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.638168 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.638177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.639558 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.641556 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.641560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.641561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.641839 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.642090 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.642091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.642091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.642225 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.642518 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.642523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.642680 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.642815 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.642964 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260810 21:00:02.643855 22506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.61@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
I20260810 21:00:02.650084 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.640421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.14.200.61:38445)
0810 21:00:02.640580 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.640591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.640859 (+ 268us) server_negotiation.cc:407] Connection header received
0810 21:00:02.640945 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.640947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.640958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.641028 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.641734 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.642764 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.643812 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.644253 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.644339 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.646925 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.646933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.646933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.646939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.648699 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.649169 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.649172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.649172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.649256 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.649467 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.649468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.649468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.649607 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.649828 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.649895 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.649972 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260810 21:00:02.651463 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.644125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40198)
0810 21:00:02.644175 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.644186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.644249 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.644596 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.644597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.644731 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.644767 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.644769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.645478 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.645479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.645925 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.645927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.646315 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.646316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.646382 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.646722 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.646727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.647885 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.650434 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.650435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.650436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.650620 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.650831 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.650832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.650832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.650892 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.651145 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.651146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.651180 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.651263 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.651356 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 21:00:02.651562 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.644194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.14.200.61:38445)
0810 21:00:02.644461 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.644462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.644469 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:02.644486 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.644487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.644495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.644593 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.644864 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.645393 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.646004 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.646233 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.646315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.647967 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.647968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.647968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.647974 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.650361 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.650695 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.650696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.650696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.650749 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.650962 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.650963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.650963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.651069 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.651139 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.651285 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.651405 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260810 21:00:02.652743 22506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Bootstrap starting.
I20260810 21:00:02.653405 22506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:02.653690 22506 log.cc:824] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:02.654412 22506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: No bootstrap required, opened a new log
I20260810 21:00:02.655802 22506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } }
I20260810 21:00:02.655931 22506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:02.655952 22506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101f8d8de7db4b6da3f9366c4c20dbdb, State: Initialized, Role: FOLLOWER
I20260810 21:00:02.656114 22506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } }
I20260810 21:00:02.656428 22506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:02.656384 22513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } } }
I20260810 21:00:02.656497 22513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: FOLLOWER
W20260810 21:00:02.658236 22524 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:00:02.658293 22524 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:00:02.659999 22431 master_runner.cc:428] Detected that this master 101f8d8de7db4b6da3f9366c4c20dbdb is joining an existing cluster
I20260810 21:00:02.660058 22431 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.200.61:38445
I20260810 21:00:02.668171 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.661023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.14.200.61:38445)
0810 21:00:02.661061 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.661063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.661310 (+ 247us) server_negotiation.cc:407] Connection header received
0810 21:00:02.661404 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.661405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.661413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.661461 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.661938 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.662635 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.663722 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.663928 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.664010 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.665123 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.665125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.665125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.665133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.666957 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.667347 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.667348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.667348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.667402 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.667615 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.667616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.667616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.667750 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.667818 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.667961 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.668064 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:00:02.668304 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.661079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40212)
0810 21:00:02.661250 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.661260 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.661331 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.661473 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.661474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.661725 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.661797 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.661799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.662723 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.662724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.663648 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.663651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.664007 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.664008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.664089 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.664542 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.664548 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.665033 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.667031 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.667033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.667033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.667251 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.667477 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.667477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.667478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.667542 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.667821 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.667822 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.667862 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.667938 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.668158 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":92,"thread_start_us":93,"threads_started":1}
I20260810 21:00:02.669333 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.661035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.14.200.62:39009)
0810 21:00:02.661078 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.661079 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.661279 (+ 200us) server_negotiation.cc:407] Connection header received
0810 21:00:02.661348 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.661349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.661360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.661415 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.661842 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.662632 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.664708 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.665049 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.665112 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.666257 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.666258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.666259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.666265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.667915 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.668380 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.668381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.668382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.668442 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.668679 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.668681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.668681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.668854 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.668910 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.669120 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.669237 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:00:02.669678 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.660978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38054)
0810 21:00:02.661220 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.661231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.661290 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.661491 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.661491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.661684 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.661745 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.661747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.663748 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.663748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.664611 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.664613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.665164 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.665164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.665232 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.665767 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.665784 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.666154 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.668012 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.668013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.668014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.668276 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.668518 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.668519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.668520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.668591 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.668962 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.668963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.669003 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.669478 (+ 475us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.669578 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":199,"threads_started":1}
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.62@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
I20260810 21:00:02.678871 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.671929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40220)
0810 21:00:02.672087 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.672126 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.672194 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.672349 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.672351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.672601 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.672646 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.672647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.673504 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.673505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.674055 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.674057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.674486 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.674487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.674542 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.674984 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.674989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.676273 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.677929 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.677935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.677937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.678122 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.678318 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.678318 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.678319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.678371 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.678614 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.678620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.678660 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.678721 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.678806 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260810 21:00:02.678929 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.671980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.14.200.61:38445)
0810 21:00:02.672031 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.672033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.672194 (+ 161us) server_negotiation.cc:407] Connection header received
0810 21:00:02.672268 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.672269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.672280 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.672347 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.672710 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.673418 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.674126 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.674412 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.674479 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.676351 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.676353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.676353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.676361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.677834 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.678193 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.678194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.678195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.678252 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.678428 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.678429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.678430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.678552 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.678617 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.678766 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.678837 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:00:02.679504 21925 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.200.61:38445: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
member_type: NON_VOTER
last_known_addr {
host: "127.14.200.61"
port: 38445
}
attrs {
promote: true
}
}
dest_uuid: "c31bf9e284f94a21866c0aaa6b293f14"
cas_config_opid_index: -1
I20260810 21:00:02.679915 21925 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } }
I20260810 21:00:02.680439 22535 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } }
I20260810 21:00:02.680738 22477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:02.681587 22535 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.200.61:38445
I20260810 21:00:02.681711 22477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 21:00:02.681912 22431 master.cc:537] Master@127.14.200.61:38445 shutting down...
I20260810 21:00:02.682183 22535 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 21:00:02.682979 21907 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 21:00:02.683499 22536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } }
I20260810 21:00:02.683597 22536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
W20260810 21:00:02.683730 21907 consensus_peers.cc:487] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 -> Peer 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61:38445): Couldn't send request to peer 101f8d8de7db4b6da3f9366c4c20dbdb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 21:00:02.684913 22431 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:02.685012 22431 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:02.685062 22431 tablet_replica.cc:330] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: stopping tablet replica
I20260810 21:00:02.687855 22431 master.cc:552] Master@127.14.200.61:38445 shutdown complete.
I20260810 21:00:02.687920 22431 master_runner.cc:305] Clearing existing system tablet
I20260810 21:00:02.688057 22431 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.688117 22431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.688717 22431 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:02.689239 22541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.689357 22431 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.689411 22431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.689436 22431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "fd198296baa68d0059b2b61cc8b2d026"
server_key_iv: "1b4a27becb17a51b9c4d9f05e13e9a38"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.690079 22431 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:02.691365 22431 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 21:00:02.691423 22431 log.cc:1196] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 21:00:02.691634 22431 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Deleting consensus metadata
I20260810 21:00:02.691761 22431 master_runner.cc:315] Copying system tablet from 127.14.200.62:39009
I20260810 21:00:02.692375 22431 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: tablet copy: Beginning tablet copy session from remote peer at address 127.14.200.62:39009
I20260810 21:00:02.698918 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.692809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.14.200.62:39009)
0810 21:00:02.692856 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.692858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.692986 (+ 128us) server_negotiation.cc:407] Connection header received
0810 21:00:02.693055 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.693055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.693063 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.693109 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.693466 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.693956 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.694999 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.695176 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.695225 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.696235 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.696236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.696237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.696243 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.697708 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.698097 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.698099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.698099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.698155 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.698363 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.698364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.698364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.698474 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.698567 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.698735 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.698828 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:00:02.698935 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.692745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38056)
0810 21:00:02.692918 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.692930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.692996 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.693132 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.693133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.693335 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.693394 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.693396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.694032 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.694033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.694916 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.694919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.695247 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.695248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:02.695321 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.695742 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:02.695748 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:02.696151 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:02.697827 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.697828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.697829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.698003 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.698231 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:02.698232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:02.698232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:02.698297 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:02.698561 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:02.698562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:02.698625 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 21:00:02.698709 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 21:00:02.698830 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260810 21:00:02.699447 21946 tablet_copy_service.cc:140] P c31bf9e284f94a21866c0aaa6b293f14: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 101f8d8de7db4b6da3f9366c4c20dbdb ({username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:38056)
I20260810 21:00:02.699546 21946 tablet_copy_service.cc:161] P c31bf9e284f94a21866c0aaa6b293f14: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 101f8d8de7db4b6da3f9366c4c20dbdb at {username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:38056: session id = 101f8d8de7db4b6da3f9366c4c20dbdb-00000000000000000000000000000000
I20260810 21:00:02.700204 21946 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 21:00:02.700816 22431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 21:00:02.701912 22431 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: tablet copy: Starting download of 0 data blocks...
I20260810 21:00:02.702077 22431 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: tablet copy: Starting download of 1 WAL segments...
I20260810 21:00:02.702945 22431 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 21:00:02.703730 21946 tablet_copy_service.cc:342] P c31bf9e284f94a21866c0aaa6b293f14: Request end of tablet copy session 101f8d8de7db4b6da3f9366c4c20dbdb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:38056
I20260810 21:00:02.703788 21946 tablet_copy_service.cc:433] P c31bf9e284f94a21866c0aaa6b293f14: ending tablet copy session 101f8d8de7db4b6da3f9366c4c20dbdb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 101f8d8de7db4b6da3f9366c4c20dbdb
I20260810 21:00:02.706503 22431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.707592 22552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.707612 22551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.707844 22554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.707902 22431 server_base.cc:1034] running on GCE node
I20260810 21:00:02.708097 22431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.708230 22431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.709348 22431 hybrid_clock.cc:648] HybridClock initialized: now 1786395602709334 us; error 31 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.710608 22431 init.cc:377] Logged in from keytab as oryx/127.14.200.61@KRBTEST.COM (short username oryx)
I20260810 21:00:02.710846 22431 webserver.cc:460] Webserver started at http://127.14.200.61:35903/ using document root <none> and password file <none>
I20260810 21:00:02.710970 22431 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.710997 22431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.711637 22431 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:02.712189 22561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.712321 22431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.712375 22431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "101f8d8de7db4b6da3f9366c4c20dbdb"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "fd198296baa68d0059b2b61cc8b2d026"
server_key_iv: "1b4a27becb17a51b9c4d9f05e13e9a38"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.712430 22431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.749400 22431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.751116 22431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.754833 22431 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.61:38445
I20260810 21:00:02.754868 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.61:38445 every 8 connection(s)
I20260810 21:00:02.755204 22431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
I20260810 21:00:02.756150 22620 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:00:02.756577 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Bootstrap starting.
I20260810 21:00:02.758968 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:00:02.759236 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Bootstrap complete.
I20260810 21:00:02.759444 22620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } }
I20260810 21:00:02.759518 22620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 101f8d8de7db4b6da3f9366c4c20dbdb, State: Initialized, Role: LEARNER
I20260810 21:00:02.759545 22620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } }
I20260810 21:00:02.759776 22620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:02.761603 22623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } }
I20260810 21:00:02.761806 22623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: LEARNER
I20260810 21:00:02.761967 22635 catalog_manager.cc:1259] Loaded cluster ID: 87e6f592ab354a309a2ebb8bb9f0c966
I20260810 21:00:02.762017 22635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: loading cluster ID for follower catalog manager: success
I20260810 21:00:02.762866 22635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: acquiring CA information for follower catalog manager: success
I20260810 21:00:02.763146 22635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:00:02.809029 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.802225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.14.200.61:38445)
0810 21:00:02.802388 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.802390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.802396 (+ 6us) server_negotiation.cc:407] Connection header received
0810 21:00:02.802421 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.802421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.802431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.802509 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.803115 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.803786 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.804570 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:02.804824 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:02.804903 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:02.806296 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:02.806298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:02.806298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:02.806306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:02.807761 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.808219 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.808219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.808220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.808292 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:02.808485 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:02.808486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:02.808486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:02.808592 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:02.808666 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:02.808846 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 21:00:02.808918 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.14.200.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.60.keytab.
Entry for principal oryx/127.14.200.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.60.keytab.
I20260810 21:00:02.839900 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:42683
--webserver_interface=127.14.200.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.60.keytab
--principal=oryx/127.14.200.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:02.948076 22641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:02.948316 22641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:02.948356 22641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:02.949780 22641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:02.949819 22641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:02.949834 22641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:02.949846 22641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:02.949858 22641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:02.951458 22641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.60.keytab
--principal=oryx/127.14.200.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.200.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:02.951678 22641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:02.951877 22641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:02.956858 22650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.956859 22646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:02.956868 22648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:02.957304 22641 server_base.cc:1034] running on GCE node
I20260810 21:00:02.957546 22641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:02.957825 22641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:02.958982 22641 hybrid_clock.cc:648] HybridClock initialized: now 1786395602958974 us; error 39 us; skew 500 ppm
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:02.961994 22641 init.cc:377] Logged in from keytab as oryx/127.14.200.60@KRBTEST.COM (short username oryx)
I20260810 21:00:02.962447 22641 webserver.cc:460] Webserver started at http://127.14.200.60:42411/ using document root <none> and password file <none>
I20260810 21:00:02.962637 22641 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:02.962677 22641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:02.962764 22641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:02.963929 22641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/instance:
uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "cd6eaa6b3094097b229a0a592804c229"
server_key_iv: "45837db210643ff09404054203335d91"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.964286 22641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal/instance:
uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "cd6eaa6b3094097b229a0a592804c229"
server_key_iv: "45837db210643ff09404054203335d91"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.965710 22641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:00:02.966539 22658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:02.966730 22641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:00:02.966835 22641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "cd6eaa6b3094097b229a0a592804c229"
server_key_iv: "45837db210643ff09404054203335d91"
server_key_version: "encryptionkey@0"
I20260810 21:00:02.966910 22641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:02.982246 22641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:02.984293 22641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:02.984452 22641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:02.994614 22641 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.60:42683
I20260810 21:00:02.994647 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.60:42683 every 8 connection(s)
I20260810 21:00:02.995066 22641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
I20260810 21:00:02.996143 22714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:02.997012 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22641
I20260810 21:00:02.997972 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.60@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
Aug 10 21:00:02 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
I20260810 21:00:03.006644 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.998981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.14.200.62:39009)
0810 21:00:02.999051 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.999053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.999068 (+ 15us) server_negotiation.cc:407] Connection header received
0810 21:00:02.999155 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.999156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.999169 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.999238 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.999732 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.000220 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.001186 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.001379 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.001487 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.003526 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.003528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.003528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.003536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.005173 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.005646 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.005647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.005648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.005706 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.005993 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.005994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.005994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.006148 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.006203 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.006477 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.006554 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 21:00:03.006824 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.998829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38058)
0810 21:00:02.998983 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:02.999023 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:02.999088 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:02.999262 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:02.999265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:02.999557 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:02.999642 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:02.999646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.000330 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.000332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.001084 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.001086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.001535 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.001536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.001617 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.002096 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.002104 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.003442 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.005275 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.005282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.005285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.005558 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.005781 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.005782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.005783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.005893 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.006208 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.006215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.006346 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.006449 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.006594 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260810 21:00:03.007346 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:00:03.008096 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:02.997624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.14.200.60:42683)
0810 21:00:02.997794 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:02.997802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:02.997827 (+ 25us) server_negotiation.cc:407] Connection header received
0810 21:00:02.998255 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:02.998262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:02.998308 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:02.998466 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:02.999214 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.000072 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.000883 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.001222 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.001349 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.003986 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.003994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.003994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.004001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.005534 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.007012 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.007015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.007015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.007075 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.007307 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.007308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.007309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.007448 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.007613 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.007851 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.007967 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.60@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
I20260810 21:00:03.014166 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.007604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40254)
0810 21:00:03.007649 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.007660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.007697 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.007861 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.007862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.008051 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.008170 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.008173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.009076 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.009077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.009554 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.009556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.009943 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.009943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.010038 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.010446 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.010451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.011541 (+ 1090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.013319 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.013319 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.013320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.013477 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.013638 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.013638 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.013639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.013709 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.013915 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.013915 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.013948 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.013993 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.014093 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 21:00:03.014194 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.007686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.14.200.61:38445)
0810 21:00:03.007727 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.007728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.007736 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:03.007760 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.007760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.007778 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.007849 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.008286 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.008989 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.009634 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.009852 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.009934 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.011615 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.011616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.011616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.011623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.013252 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.013537 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.013537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.013537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.013584 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.013762 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.013763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.013763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.013847 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.013924 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.014048 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.014134 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:00:03.014669 22714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395602, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.60@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
I20260810 21:00:03.020460 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.014973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60168)
0810 21:00:03.015031 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.015040 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.015075 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.015227 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.015227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.015332 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.015357 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.015358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.015948 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.015948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.016391 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.016392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.016720 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.016720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.016758 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.017191 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.017196 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.018249 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.019497 (+ 1248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.019498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.019498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.019652 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.019848 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.019849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.019849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.019902 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.020144 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.020145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.020179 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.020260 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.020348 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 21:00:03.020511 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.015027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.14.200.60:42683)
0810 21:00:03.015064 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.015066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.015074 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:03.015150 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.015151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.015159 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.015214 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.015415 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.015844 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.016454 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.016644 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.016691 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.018335 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.018336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.018336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.018343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.019408 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.019736 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.019737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.019737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.019788 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.019971 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.019972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.019973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.020073 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.020139 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.020280 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.020400 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":16}
I20260810 21:00:03.022050 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Bootstrap starting.
I20260810 21:00:03.022833 22714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:03.023262 22714 log.cc:824] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:03.023976 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: No bootstrap required, opened a new log
I20260810 21:00:03.026286 22714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } }
I20260810 21:00:03.026407 22714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:03.026448 22714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9773b1ee71d242ad9dcb5629f5ad6d9e, State: Initialized, Role: FOLLOWER
I20260810 21:00:03.026535 22714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } }
I20260810 21:00:03.026903 22714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:03.026901 22721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } } }
I20260810 21:00:03.027101 22721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: FOLLOWER
W20260810 21:00:03.029575 22735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:00:03.029634 22735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:00:03.032022 22641 master_runner.cc:428] Detected that this master 9773b1ee71d242ad9dcb5629f5ad6d9e is joining an existing cluster
I20260810 21:00:03.032086 22641 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.200.60:42683
I20260810 21:00:03.042016 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.034257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60178)
0810 21:00:03.034826 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.034836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.034912 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.035095 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.035096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.035481 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.035641 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.035643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036505 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.036506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.037672 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.037687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038158 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.038251 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.038718 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.038723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.039156 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.041020 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.041020 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.041021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.041181 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.041382 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.041383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.041383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.041438 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.041733 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.041733 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.041766 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.041812 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.041910 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":7,"thread_start_us":172,"threads_started":1}
I20260810 21:00:03.042093 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.034227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.14.200.60:42683)
0810 21:00:03.034273 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.034275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.034881 (+ 606us) server_negotiation.cc:407] Connection header received
0810 21:00:03.034995 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.034996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.035005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.035031 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.035719 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036419 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.037780 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038044 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038080 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.039278 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.039280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.039280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.039287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.040922 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.041250 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.041251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.041252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.041313 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.041501 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.041502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.041502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.041652 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.041736 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.041861 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.041980 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":28}
I20260810 21:00:03.059963 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21991
I20260810 21:00:03.067375 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.034493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.14.200.61:38445)
0810 21:00:03.034564 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.034566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.034575 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:03.034609 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.034610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.034621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.034678 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.035512 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036368 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.037729 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038052 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038133 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.057933 (+ 19800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.057935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.057936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.057945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.059468 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.059871 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.059872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.059872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.059960 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.066745 (+ 6785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.066747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.066747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.066904 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.066958 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.067178 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.067272 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 21:00:03.067466 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.033790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.14.200.62:39009)
0810 21:00:03.033831 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.033833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.034538 (+ 705us) server_negotiation.cc:407] Connection header received
0810 21:00:03.034767 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.034768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.034787 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.034830 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.035696 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036437 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.037820 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038096 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038133 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.058017 (+ 19884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.058019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.058020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.058028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.060258 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.066329 (+ 6071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.066330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.066331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.066406 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.066671 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.066672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.066672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.066854 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.066937 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.067195 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.067300 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:00:03.067689 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.033716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40268)
0810 21:00:03.034054 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.034066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.034118 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.034752 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.034753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.035165 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.035406 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.035409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036468 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.036468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.037624 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.037627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038223 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.038316 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.057232 (+ 18916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.057238 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.057808 (+ 570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.059558 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.059559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.059559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.059787 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.066588 (+ 6801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.066589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.066590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.066658 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.067007 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.067008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.067068 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.067465 (+ 397us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.067580 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":252,"threads_started":1}
I20260810 21:00:03.068740 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.033764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38060)
0810 21:00:03.034471 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.034482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.034629 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.034904 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.034904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.035345 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.035569 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.035571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.036518 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.036519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.037733 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.037735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.038196 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.038197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.038297 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.057394 (+ 19097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.057399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.057828 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.065941 (+ 8113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.065942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.065943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.066219 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.066508 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.066509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.066510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.066582 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.066978 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.066979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.067040 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.068401 (+ 1361us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.068561 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":936,"threads_started":1}
I20260810 21:00:03.075073 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:42973
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:39009
--webserver_port=34697
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395601, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.62@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
I20260810 21:00:03.080508 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.072361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60190)
0810 21:00:03.072422 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.072435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.072492 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.072613 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.072614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.072820 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.072874 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.072877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.073765 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.073766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.074645 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.074649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.075157 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.075158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.075248 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.075733 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.075739 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.077409 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.079352 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.079353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.079354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.079577 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.079792 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.079793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.079793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.079862 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.080173 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.080174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.080221 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.080306 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.080407 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 21:00:03.080856 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.072462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.14.200.60:42683)
0810 21:00:03.072496 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.072497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.072505 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:03.072545 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.072546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.072555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.072610 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.072959 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.073675 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.074744 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.075061 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.075133 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.077503 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.077505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.077505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.077514 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.079261 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.079648 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.079650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.079650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.079713 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.079942 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.079943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.079944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.080098 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.080177 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.080351 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.080434 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:00:03.081454 21925 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.200.60:42683: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
member_type: NON_VOTER
last_known_addr {
host: "127.14.200.60"
port: 42683
}
attrs {
promote: true
}
}
dest_uuid: "c31bf9e284f94a21866c0aaa6b293f14"
cas_config_opid_index: 5
I20260810 21:00:03.081709 21925 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } }
I20260810 21:00:03.082077 22536 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.082540 22687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:03.083062 22536 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.200.60:42683
I20260810 21:00:03.083123 22536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.083186 22536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.083668 22687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 21:00:03.083992 22535 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:03.084220 22641 master.cc:537] Master@127.14.200.60:42683 shutting down...
W20260810 21:00:03.084839 21907 consensus_peers.cc:487] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 -> Peer 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60:42683): Couldn't send request to peer 9773b1ee71d242ad9dcb5629f5ad6d9e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 21:00:03.085510 22721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:03.085598 22721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } }
I20260810 21:00:03.085810 22721 leader_election.cc:288] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009), 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61:38445)
I20260810 21:00:03.086056 22591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" is_pre_election: true
I20260810 21:00:03.086160 22591 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Handling vote request from an unknown peer 9773b1ee71d242ad9dcb5629f5ad6d9e
I20260810 21:00:03.086202 22591 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 9773b1ee71d242ad9dcb5629f5ad6d9e for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260810 21:00:03.086158 21936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c31bf9e284f94a21866c0aaa6b293f14" is_pre_election: true
I20260810 21:00:03.086469 22662 leader_election.cc:302] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9773b1ee71d242ad9dcb5629f5ad6d9e; no voters: 101f8d8de7db4b6da3f9366c4c20dbdb, c31bf9e284f94a21866c0aaa6b293f14
I20260810 21:00:03.086644 22721 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:00:03.089973 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.082173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40282)
0810 21:00:03.083660 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.083671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.083706 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.084033 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.084034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.084036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.084072 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.084073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.085819 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.085820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.087005 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.087009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.089715 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.089716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.089784 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.089806 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.089856 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.089893 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471}
I20260810 21:00:03.090240 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.082277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.14.200.61:38445)
0810 21:00:03.083821 (+ 1544us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.083823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.083831 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:03.083854 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.083855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.083879 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.083971 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.084717 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.085588 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.088851 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.089472 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.089544 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.089614 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.090133 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.090179 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1519}
I20260810 21:00:03.090413 22591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 21:00:03.090577 22535 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 21:00:03.091101 22591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.091450 22641 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:03.091526 22641 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:03.091564 22641 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: stopping tablet replica
I20260810 21:00:03.092154 22623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c31bf9e284f94a21866c0aaa6b293f14. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.092234 22623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: LEARNER
I20260810 21:00:03.092263 22623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: NON_VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: true } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.092303 22623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: LEARNER
I20260810 21:00:03.092747 22535 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: attempting to promote NON_VOTER 101f8d8de7db4b6da3f9366c4c20dbdb to VOTER
I20260810 21:00:03.093108 22535 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } }
I20260810 21:00:03.093568 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.093421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60202)
0810 21:00:03.093486 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.093532 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 21:00:03.093860 22591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 LEARNER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 21:00:03.093920 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.093844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60216)
0810 21:00:03.093879 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.093899 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 21:00:03.094105 21907 consensus_peers.cc:487] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 -> Peer 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60:42683): Couldn't send request to peer 9773b1ee71d242ad9dcb5629f5ad6d9e. Status: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42683: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 21:00:03.094202 22537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 21:00:03.095045 22537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.095981 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.096053 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.096104 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.096143 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.096349 22591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.097465 22746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.097554 22746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:00:03.105810 22641 master.cc:552] Master@127.14.200.60:42683 shutdown complete.
I20260810 21:00:03.105865 22641 master_runner.cc:305] Clearing existing system tablet
I20260810 21:00:03.106011 22641 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:03.106058 22641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:03.106743 22641 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260810 21:00:03.107273 22748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.107436 22641 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:03.107499 22641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.107522 22641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "cd6eaa6b3094097b229a0a592804c229"
server_key_iv: "45837db210643ff09404054203335d91"
server_key_version: "encryptionkey@0"
I20260810 21:00:03.108206 22641 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:03.109591 22641 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 21:00:03.109661 22641 log.cc:1196] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 21:00:03.109846 22641 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Deleting consensus metadata
I20260810 21:00:03.110193 22641 master_runner.cc:315] Copying system tablet from 127.14.200.62:39009
I20260810 21:00:03.110903 22641 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: tablet copy: Beginning tablet copy session from remote peer at address 127.14.200.62:39009
I20260810 21:00:03.118674 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.111228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38068)
0810 21:00:03.111389 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.111415 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.111479 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.111691 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.111692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.111923 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.111971 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.111973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.113041 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.113042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.113880 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.113884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.114392 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.114394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.114454 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.115037 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.115043 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.115441 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.117473 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.117474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.117475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.117716 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.117978 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.117978 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.117979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.118046 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.118336 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.118338 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.118397 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.118469 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.118572 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260810 21:00:03.118687 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.111452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.14.200.62:39009)
0810 21:00:03.111523 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.111526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.111535 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:03.111562 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.111562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.111573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.111657 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.112098 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.112940 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.114019 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.114301 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.114380 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.115545 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.115547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.115548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.115556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.117371 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.117812 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.117813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.117814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.117893 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.118121 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.118122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.118123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.118257 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.118327 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.118516 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.118597 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 21:00:03.119341 21946 tablet_copy_service.cc:140] P c31bf9e284f94a21866c0aaa6b293f14: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9773b1ee71d242ad9dcb5629f5ad6d9e ({username='oryx', principal='oryx/127.14.200.60@KRBTEST.COM'} at 127.0.0.1:38068)
I20260810 21:00:03.119403 21946 tablet_copy_service.cc:161] P c31bf9e284f94a21866c0aaa6b293f14: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9773b1ee71d242ad9dcb5629f5ad6d9e at {username='oryx', principal='oryx/127.14.200.60@KRBTEST.COM'} at 127.0.0.1:38068: session id = 9773b1ee71d242ad9dcb5629f5ad6d9e-00000000000000000000000000000000
I20260810 21:00:03.120040 21946 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 21:00:03.120550 22641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 21:00:03.121901 22641 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: tablet copy: Starting download of 0 data blocks...
I20260810 21:00:03.122056 22641 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: tablet copy: Starting download of 1 WAL segments...
I20260810 21:00:03.122997 22641 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 21:00:03.123983 21946 tablet_copy_service.cc:342] P c31bf9e284f94a21866c0aaa6b293f14: Request end of tablet copy session 9773b1ee71d242ad9dcb5629f5ad6d9e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.200.60@KRBTEST.COM'} at 127.0.0.1:38068
I20260810 21:00:03.124040 21946 tablet_copy_service.cc:433] P c31bf9e284f94a21866c0aaa6b293f14: ending tablet copy session 9773b1ee71d242ad9dcb5629f5ad6d9e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9773b1ee71d242ad9dcb5629f5ad6d9e
I20260810 21:00:03.127221 22641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:03.128388 22755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.129696 22756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.129770 22758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:03.129945 22641 server_base.cc:1034] running on GCE node
I20260810 21:00:03.130154 22641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:03.130318 22641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:03.131460 22641 hybrid_clock.cc:648] HybridClock initialized: now 1786395603131453 us; error 27 us; skew 500 ppm
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:03.133087 22641 init.cc:377] Logged in from keytab as oryx/127.14.200.60@KRBTEST.COM (short username oryx)
I20260810 21:00:03.133347 22641 webserver.cc:460] Webserver started at http://127.14.200.60:35085/ using document root <none> and password file <none>
I20260810 21:00:03.133466 22641 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:03.133505 22641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:03.134279 22641 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:00:03.134932 22766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.135083 22641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.135138 22641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "cd6eaa6b3094097b229a0a592804c229"
server_key_iv: "45837db210643ff09404054203335d91"
server_key_version: "encryptionkey@0"
I20260810 21:00:03.135191 22641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:03.160835 22641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:03.162731 22641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:03.166599 22641 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.60:42683
I20260810 21:00:03.166716 22821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.60:42683 every 8 connection(s)
I20260810 21:00:03.166967 22641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
I20260810 21:00:03.167902 22822 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:00:03.169094 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Bootstrap starting.
I20260810 21:00:03.172250 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:00:03.172477 22822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: Bootstrap complete.
I20260810 21:00:03.172703 22822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } }
I20260810 21:00:03.172820 22822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9773b1ee71d242ad9dcb5629f5ad6d9e, State: Initialized, Role: LEARNER
I20260810 21:00:03.172868 22822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } }
I20260810 21:00:03.173055 22822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:03.173081 22826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.173290 22826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: LEARNER
I20260810 21:00:03.174566 22839 catalog_manager.cc:1259] Loaded cluster ID: 87e6f592ab354a309a2ebb8bb9f0c966
I20260810 21:00:03.174659 22839 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: loading cluster ID for follower catalog manager: success
I20260810 21:00:03.176649 22839 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: acquiring CA information for follower catalog manager: success
I20260810 21:00:03.176998 22839 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 21:00:03.177645 22744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:03.177831 22744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:03.177855 22744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:03.179136 22744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:03.179172 22744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:03.179199 22744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:03.180699 22744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:42973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=34697
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:03.181020 22744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:03.181258 22744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:03.187440 22845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.187407 22846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.187453 22848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:03.187763 22744 server_base.cc:1034] running on GCE node
I20260810 21:00:03.187994 22744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:03.188246 22744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:03.189493 22744 hybrid_clock.cc:648] HybridClock initialized: now 1786395603189470 us; error 33 us; skew 500 ppm
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:03.193267 22744 init.cc:377] Logged in from keytab as oryx/127.14.200.1@KRBTEST.COM (short username oryx)
I20260810 21:00:03.193794 22744 webserver.cc:460] Webserver started at http://127.14.200.1:34697/ using document root <none> and password file <none>
I20260810 21:00:03.194041 22744 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:03.194124 22744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:03.196126 22744 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:03.196818 22855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.197031 22744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:03.197130 22744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "d9ee17ce95c946ebbce389522fe34c40"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "a9c0cf458e15cee5d534224207951e3c"
server_key_iv: "0a9b7c2bf7562fe25a8827b4bf75fd6e"
server_key_version: "encryptionkey@0"
I20260810 21:00:03.197600 22744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:03.214607 22744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:03.217056 22744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:03.217195 22744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:03.217393 22744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:03.217643 22744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:03.217674 22744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.217695 22744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:03.217708 22744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.233359 22744 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42973
I20260810 21:00:03.233507 22978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42973 every 8 connection(s)
I20260810 21:00:03.233850 22744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:03.241843 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22744
I20260810 21:00:03.241999 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22136
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
I20260810 21:00:03.248201 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.235362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:42373 (local address 127.14.200.61:38445)
0810 21:00:03.235419 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.235421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.235580 (+ 159us) server_negotiation.cc:407] Connection header received
0810 21:00:03.236810 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.236812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.236826 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.236875 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.237823 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.238586 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240351 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240603 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240680 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.244052 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.244053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.244054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.244063 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.246384 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.247052 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.247053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.247054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.247110 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.247400 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.247402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.247402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.247572 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.247613 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.247972 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.248075 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:00:03.248418 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.235151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:55473 (local address 127.14.200.62:39009)
0810 21:00:03.235210 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.235211 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.235338 (+ 127us) server_negotiation.cc:407] Connection header received
0810 21:00:03.236722 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.236724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.236738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.236805 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.237823 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.238573 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240413 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240739 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240874 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.244470 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.244471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.244472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.244480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.246294 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.246732 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.246734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.246734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.246800 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.247401 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.247403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.247403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.247550 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.247597 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.248254 (+ 657us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.248337 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:00:03.248750 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.235183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.14.200.1:42373)
0810 21:00:03.235498 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.235507 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.236578 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.236894 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.236898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.237460 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.237701 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.237704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.239215 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.239216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.240255 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.240262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240680 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.240838 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.241507 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.241520 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.243946 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.246747 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.246748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.246748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.246959 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.247198 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.247199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.247199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.247299 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.247660 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.247660 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.247813 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.248307 (+ 494us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.248477 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":11,"thread_start_us":98,"threads_started":1}
I20260810 21:00:03.249382 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.234792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:55473)
0810 21:00:03.235264 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.235275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.236607 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.236894 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.236898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.237461 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.237701 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.237704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.238753 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.238753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.240281 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.240283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240882 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.241436 (+ 553us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.242024 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.242051 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.244358 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.246391 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.246399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.246402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.246649 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.246892 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.246893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.246894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.247299 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.247643 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.247650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.247994 (+ 344us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.249108 (+ 1114us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.249287 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":44,"thread_start_us":118,"threads_started":1}
I20260810 21:00:03.250466 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.234781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37599 (local address 127.14.200.60:42683)
0810 21:00:03.234994 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.234996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.235252 (+ 256us) server_negotiation.cc:407] Connection header received
0810 21:00:03.236848 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.236850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.236865 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.236997 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.237824 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.238522 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240132 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240388 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240452 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.246522 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.246523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.246524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.246533 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.249009 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.249559 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.249560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.249560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.249654 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.249897 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.249899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.249899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.250015 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.250117 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.250331 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.250402 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":60,"threads_started":1}
I20260810 21:00:03.250550 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.234789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.14.200.1:37599)
0810 21:00:03.235098 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.235144 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.236576 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.237003 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.237004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.237311 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.237643 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.237649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.238718 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.238719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.240033 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.240036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.240458 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.240459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.240708 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.241430 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.241438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.246398 (+ 4960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.249139 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.249140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.249141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.249452 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.249746 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.249746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.249747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.249820 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.250116 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.250118 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.250176 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.250269 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.250396 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":76,"thread_start_us":170,"threads_started":1}
I20260810 21:00:03.251063 22979 heartbeater.cc:347] Connected to a master server at 127.14.200.60:42683
I20260810 21:00:03.251152 22979 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.251355 22979 heartbeater.cc:511] Master 127.14.200.60:42683 requested a full tablet report, sending...
I20260810 21:00:03.251885 22775 ts_manager.cc:194] Registered new tserver with Master: d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:03.255749 22981 heartbeater.cc:347] Connected to a master server at 127.14.200.61:38445
I20260810 21:00:03.255810 22981 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.255901 22981 heartbeater.cc:511] Master 127.14.200.61:38445 requested a full tablet report, sending...
I20260810 21:00:03.256387 22980 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:03.256381 22581 ts_manager.cc:194] Registered new tserver with Master: d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:03.256444 22980 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.256511 22980 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:03.256866 21925 ts_manager.cc:194] Re-registered known tserver with Master: d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:03.257398 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:55473
I20260810 21:00:03.260763 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:43259
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:39009
--webserver_port=38567
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:03.346194 22989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:03.346390 22989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:03.346413 22989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:03.347525 22989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:03.347563 22989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:03.347589 22989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:03.349187 22989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:43259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=38567
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:03.349476 22989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:03.349726 22989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:03.355314 22996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.355314 22998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.355350 22995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:03.355736 22989 server_base.cc:1034] running on GCE node
I20260810 21:00:03.355963 22989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:03.356233 22989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:03.357436 22989 hybrid_clock.cc:648] HybridClock initialized: now 1786395603357422 us; error 24 us; skew 500 ppm
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:03.360029 22989 init.cc:377] Logged in from keytab as oryx/127.14.200.2@KRBTEST.COM (short username oryx)
I20260810 21:00:03.360587 22989 webserver.cc:460] Webserver started at http://127.14.200.2:38567/ using document root <none> and password file <none>
I20260810 21:00:03.360900 22989 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:03.360993 22989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:03.362391 22989 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:03.362996 23005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.363180 22989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:03.363281 22989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "08ffe0f6df39453ea94a10b8630e582c"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "4c7fbea38cf67089216d27c33adbb6d6"
server_key_iv: "d40446f2700dd1ada97b7f49548b1f34"
server_key_version: "encryptionkey@0"
I20260810 21:00:03.363662 22989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:03.375298 22989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:03.377331 22989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:03.377491 22989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:03.377728 22989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:03.378093 22989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:03.378202 22989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.378330 22989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:03.378376 22989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.386099 22989 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:43259
I20260810 21:00:03.386205 23126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:43259 every 8 connection(s)
I20260810 21:00:03.386734 22989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
I20260810 21:00:03.396986 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 22989
I20260810 21:00:03.397166 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22284
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
I20260810 21:00:03.401124 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.387835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:56885 (local address 127.14.200.62:39009)
0810 21:00:03.387898 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.387900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.388547 (+ 647us) server_negotiation.cc:407] Connection header received
0810 21:00:03.389867 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.389869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.389882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.389945 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.390803 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.391602 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393864 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.394148 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.394225 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.396908 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.396910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.396910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.396919 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.399123 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.399719 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.399721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.399721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.399807 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.400117 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.400118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.400119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.400315 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.400422 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.400848 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.400962 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 21:00:03.401329 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.387725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:56885)
0810 21:00:03.388421 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.388432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.389750 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.390017 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.390026 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.390348 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.390705 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.390709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.391904 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.391905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.393510 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.393513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.394225 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.394226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.394396 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.395060 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.395064 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.396799 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.399243 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.399252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.399270 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.399609 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.399893 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.399894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.399894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.400023 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.400420 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.400429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.400607 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.400725 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.400935 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":122,"thread_start_us":496,"threads_started":1}
I20260810 21:00:03.402542 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.387949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:51099 (local address 127.14.200.60:42683)
0810 21:00:03.387987 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.387989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.388557 (+ 568us) server_negotiation.cc:407] Connection header received
0810 21:00:03.389897 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.389899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.389918 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.389956 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.390975 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.391721 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393597 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.393893 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393953 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.398175 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.398177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.398177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.398187 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.400830 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.401564 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.401565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.401565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.401633 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.401856 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.401857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.401858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.402026 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.402087 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.402298 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.402415 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:00:03.402573 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.387725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.14.200.2:51099)
0810 21:00:03.388341 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.388411 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.389773 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.390035 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.390036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.390595 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.390867 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.390870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.391915 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.391915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.393511 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.393513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.393982 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.394088 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.394838 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.394849 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.398060 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.400914 (+ 2854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.400914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.400915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.401117 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.401707 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.401708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.401708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.401800 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.402115 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.402116 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.402158 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.402218 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.402331 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":63,"thread_start_us":428,"threads_started":1}
I20260810 21:00:03.405402 23129 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:03.405520 23129 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.405737 23129 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:03.405856 23127 heartbeater.cc:347] Connected to a master server at 127.14.200.60:42683
I20260810 21:00:03.405922 23127 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.405998 23127 heartbeater.cc:511] Master 127.14.200.60:42683 requested a full tablet report, sending...
I20260810 21:00:03.406229 21925 ts_manager.cc:194] Re-registered known tserver with Master: 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259)
I20260810 21:00:03.406641 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:56885
I20260810 21:00:03.409389 22775 ts_manager.cc:194] Registered new tserver with Master: 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259)
I20260810 21:00:03.411550 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:42721
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:39009
--webserver_port=35119
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:00:03.412719 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.388510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:34665 (local address 127.14.200.61:38445)
0810 21:00:03.388563 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.388565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.388840 (+ 275us) server_negotiation.cc:407] Connection header received
0810 21:00:03.389932 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.389933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.389947 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.389983 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.391738 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.392233 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393454 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.393728 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.409592 (+ 15799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.409594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.409595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.409606 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.411316 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.411791 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.411793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.411793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.411865 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.412102 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.412103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.412104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.412240 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.412293 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.412527 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.412598 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:00:03.412889 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.388418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.14.200.2:34665)
0810 21:00:03.388745 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.388754 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.389775 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.390064 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.390064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.390598 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.390867 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.390870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.392323 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.392325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.393366 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.393369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.393811 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.393812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.394034 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.394843 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.394851 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.409421 (+ 14570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.411419 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.411420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.411421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.411674 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.411949 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.411949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.411950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.412021 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.412320 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.412321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.412385 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.412458 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.412609 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":6,"thread_start_us":176,"threads_started":1}
I20260810 21:00:03.413327 23131 heartbeater.cc:347] Connected to a master server at 127.14.200.61:38445
I20260810 21:00:03.413394 23131 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.413486 23131 heartbeater.cc:511] Master 127.14.200.61:38445 requested a full tablet report, sending...
I20260810 21:00:03.413821 22581 ts_manager.cc:194] Registered new tserver with Master: 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259)
W20260810 21:00:03.496182 23137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:03.496397 23137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:03.496420 23137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:03.497710 23137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:03.497758 23137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:03.497786 23137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:03.499377 23137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:42721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=35119
--tserver_master_addrs=127.14.200.62:39009,127.14.200.61:38445,127.14.200.60:42683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:03.499639 23137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:03.499893 23137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:03.505429 23143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.505455 23142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:03.505455 23145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:03.506007 23137 server_base.cc:1034] running on GCE node
I20260810 21:00:03.506230 23137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:03.506493 23137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:03.507639 23137 hybrid_clock.cc:648] HybridClock initialized: now 1786395603507620 us; error 40 us; skew 500 ppm
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:03.510193 23137 init.cc:377] Logged in from keytab as oryx/127.14.200.3@KRBTEST.COM (short username oryx)
I20260810 21:00:03.510582 23137 webserver.cc:460] Webserver started at http://127.14.200.3:35119/ using document root <none> and password file <none>
I20260810 21:00:03.510790 23137 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:03.510842 23137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:03.512480 23137 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:03.513132 23152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.513381 23137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:03.513485 23137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "89564e8aaf4f48f49a2b09b76363d9ca"
format_stamp: "Formatted at 2026-08-10 21:00:02 on dist-test-slave-1x4g"
server_key: "675e548e36f65b7d7e958dcc71c4c264"
server_key_iv: "51b54c83d69e32983443c83206c7465f"
server_key_version: "encryptionkey@0"
I20260810 21:00:03.513855 23137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:03.531665 23137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:03.533149 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.530336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60220)
0810 21:00:03.530412 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.530428 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.530490 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.530721 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.530723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.530725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.530786 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.530788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.531440 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.531441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.532237 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.532240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.532899 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.532900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.532973 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.532977 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.533030 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.533075 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 21:00:03.533149 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.530451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.14.200.60:42683)
0810 21:00:03.530537 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.530539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.530551 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:00:03.530583 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.530584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.530616 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.530716 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.530863 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.531349 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.532337 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.532735 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.532895 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.532979 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.533034 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.533076 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 21:00:03.533480 22795 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 21:00:03.533666 22826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c31bf9e284f94a21866c0aaa6b293f14. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.533806 22826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: LEARNER
I20260810 21:00:03.534279 23137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:03.534453 23137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:03.534710 23137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:03.535048 23137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:03.535110 23137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.535152 23137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:03.535179 23137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:03.541213 23137 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:42721
I20260810 21:00:03.541306 23278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:42721 every 8 connection(s)
I20260810 21:00:03.541656 23137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:03.547765 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 23137
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.60@KRBTEST.COM
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
Aug 10 21:00:03 dist-test-slave-1x4g krb5kdc[21874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395603, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.61@KRBTEST.COM
I20260810 21:00:03.555467 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.14.200.3:37191)
0810 21:00:03.542902 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.542911 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.543773 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.544046 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.544051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.544431 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.544597 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.544602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545602 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.545602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.546582 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.546584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.547312 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.547313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.547452 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.548241 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.548246 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.551150 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.553542 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.553551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.553553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.553894 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.554121 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.554122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.554122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.554260 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.554559 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.554567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.554758 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.554880 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.555087 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":91,"thread_start_us":62,"threads_started":1}
I20260810 21:00:03.556743 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:37191 (local address 127.14.200.61:38445)
0810 21:00:03.542983 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.542985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.542994 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:03.543895 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.543897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.543910 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.543943 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.544706 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545497 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546692 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.546888 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.551247 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.551249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.551249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.551256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.553082 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.553988 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.553990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.553990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.554052 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.554327 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.554329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.554329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.554482 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.554570 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.556534 (+ 1964us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.556650 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 21:00:03.556905 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:38577)
0810 21:00:03.542635 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.542680 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.543772 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.544046 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.544051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.544435 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.544597 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.544602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545602 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.545602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.546580 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.546583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.547056 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.547057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.547138 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.547798 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.547803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.551238 (+ 3435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.554875 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.554876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.554876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.555814 (+ 938us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.556045 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.556046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.556046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.556125 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.556472 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.556474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.556518 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.556586 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.556701 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":58,"thread_start_us":115,"threads_started":1}
I20260810 21:00:03.557462 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38577 (local address 127.14.200.62:39009)
0810 21:00:03.542430 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.542432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.542806 (+ 374us) server_negotiation.cc:407] Connection header received
0810 21:00:03.543895 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.543896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.543910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.543943 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.544705 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545497 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546677 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.546991 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.547041 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.551744 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.551745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.551746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.551753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.554781 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.555893 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.555894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.555895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.555952 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.556211 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.556212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.556213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.556383 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.556479 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.557262 (+ 783us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.557342 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:00:03.557658 23282 heartbeater.cc:347] Connected to a master server at 127.14.200.61:38445
I20260810 21:00:03.558060 23281 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39009
I20260810 21:00:03.558161 23281 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.558403 23281 heartbeater.cc:511] Master 127.14.200.62:39009 requested a full tablet report, sending...
I20260810 21:00:03.558636 23282 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.558717 23282 heartbeater.cc:511] Master 127.14.200.61:38445 requested a full tablet report, sending...
I20260810 21:00:03.558856 21925 ts_manager.cc:194] Re-registered known tserver with Master: 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3:42721)
I20260810 21:00:03.558988 22581 ts_manager.cc:194] Registered new tserver with Master: 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3:42721)
I20260810 21:00:03.559358 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.549811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.14.200.60:42683)
0810 21:00:03.549942 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.549944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.550433 (+ 489us) server_negotiation.cc:407] Connection header received
0810 21:00:03.550526 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.550527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.550540 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.550595 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.551443 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.552233 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.553415 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.553777 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.553822 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.555821 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.555822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.555823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.555830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.557820 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.558334 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.558336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.558336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.558402 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.558656 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.558657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.558657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.558785 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.558854 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.559174 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.559262 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":97,"threads_started":1}
I20260810 21:00:03.559353 21925 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:38577
I20260810 21:00:03.561851 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.549534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.14.200.61:38445)
0810 21:00:03.549699 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.549701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.549770 (+ 69us) server_negotiation.cc:407] Connection header received
0810 21:00:03.549831 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.549831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.549844 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.549911 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.550914 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.551662 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.552883 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.553227 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.553288 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.555838 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.555840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.555840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.555847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.560547 (+ 4700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.560965 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.560967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.560967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.561029 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.561252 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.561253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.561254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.561432 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.561540 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.561652 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.561734 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260810 21:00:03.562033 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:54215 (local address 127.14.200.60:42683)
0810 21:00:03.542432 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.542434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.542795 (+ 361us) server_negotiation.cc:407] Connection header received
0810 21:00:03.543871 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.543872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.543888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.543965 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.544694 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545242 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546265 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.546461 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546545 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.552143 (+ 5598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.552145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.552145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.552154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.554447 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.560980 (+ 6533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.560981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.560981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.561042 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.561240 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.561241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.561242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.561428 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.561513 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.561817 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.561924 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:00:03.562453 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.542412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.14.200.3:54215)
0810 21:00:03.542706 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.542717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.543770 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.544046 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.544049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.544370 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:03.544595 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.544599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.545357 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.545358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.546171 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.546173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.546558 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.546559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.546730 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.547367 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:03.547376 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:03.551608 (+ 4232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:03.560645 (+ 9037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.560645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.560646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.560885 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.561128 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:03.561129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:03.561129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:03.561184 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:03.561548 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:03.561549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:03.561599 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.562150 (+ 551us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.562281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":6,"thread_start_us":137,"threads_started":1}
I20260810 21:00:03.562942 23279 heartbeater.cc:347] Connected to a master server at 127.14.200.60:42683
I20260810 21:00:03.562990 23279 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:03.563041 23279 heartbeater.cc:511] Master 127.14.200.60:42683 requested a full tablet report, sending...
I20260810 21:00:03.563377 22775 ts_manager.cc:194] Registered new tserver with Master: 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3:42721)
I20260810 21:00:03.565047 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.550155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.14.200.62:39009)
0810 21:00:03.550571 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.550573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.550580 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:03.550601 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.550602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.550612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.555108 (+ 4496us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.556714 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.557949 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.559189 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.559523 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.559588 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.561192 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.561193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.561193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.561199 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.563859 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.564221 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.564222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.564222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.564287 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.564491 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.564492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.564492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.564613 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.564727 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.564862 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.564943 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":132,"threads_started":1}
I20260810 21:00:03.566697 21925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:00:03.568531 21925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:03.569355 22535 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: NON_VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: true } } }
I20260810 21:00:03.569453 22535 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.570781 22536 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: attempting to promote NON_VOTER 9773b1ee71d242ad9dcb5629f5ad6d9e to VOTER
I20260810 21:00:03.571115 22536 consensus_queue.cc:237] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } }
I20260810 21:00:03.571534 22591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 21:00:03.571768 22794 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 LEARNER]: Refusing update from remote peer c31bf9e284f94a21866c0aaa6b293f14: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 21:00:03.571768 22536 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260810 21:00:03.572088 22535 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 21:00:03.572458 23301 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.572594 22591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.573164 22795 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.573700 23301 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.573720 22746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.573773 23301 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.573789 22746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:00:03.573858 22536 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.573972 22536 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.573866 23301 sys_catalog.cc:455] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.574324 23301 sys_catalog.cc:458] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:03.574519 22826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c31bf9e284f94a21866c0aaa6b293f14" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:03.574599 22826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:00:03.579653 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43259 (local address 127.0.0.1:56248)
0810 21:00:03.575133 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.575145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.575185 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.575776 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.575777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.575779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.575828 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.575830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.577388 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.577389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.578473 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.578476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.579420 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.579421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.579478 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.579482 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.579547 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.579585 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 21:00:03.579742 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.14.200.2:43259)
0810 21:00:03.575571 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.575577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.575589 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:00:03.575662 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.575668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.575682 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.575779 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.575948 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.577293 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.578559 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.579210 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.579285 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.579484 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.579565 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.579608 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":166,"threads_started":1}
I20260810 21:00:03.580173 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:42721 (local address 127.0.0.1:50594)
0810 21:00:03.576136 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.576148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.576264 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.576552 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.576553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.576555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.576627 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.576628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.577797 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.577799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.578711 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.578713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.579948 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.579949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.579993 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.579996 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.580070 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.580108 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":654,"threads_started":1}
I20260810 21:00:03.580245 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.14.200.3:42721)
0810 21:00:03.575831 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.575836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.576242 (+ 406us) server_negotiation.cc:407] Connection header received
0810 21:00:03.576391 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.576394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.576409 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.576487 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.576701 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.577716 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.578784 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.579729 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.579799 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.579916 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.580086 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.580123 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":222,"threads_started":1}
I20260810 21:00:03.580955 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42973 (local address 127.0.0.1:37544)
0810 21:00:03.576188 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.576206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.576260 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.576479 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.576480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.576482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.576532 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.576534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.578083 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.578084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.579125 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.579129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.580685 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.580686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.580733 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.580737 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.580842 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.580883 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"thread_start_us":873,"threads_started":1}
I20260810 21:00:03.581120 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.575382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37544 (local address 127.14.200.1:42973)
0810 21:00:03.575687 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.575692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.576263 (+ 571us) server_negotiation.cc:407] Connection header received
0810 21:00:03.576372 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.576376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.576396 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.576482 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.576588 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.577979 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.579221 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.580424 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.580492 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.580653 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.580868 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.580910 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":166,"threads_started":1}
I20260810 21:00:03.581020 23205 tablet_service.cc:1467] Processing CreateTablet for tablet dedc2c023566419baac83160511d3efb (DEFAULT_TABLE table=test-table [id=7b8c1a85ef3143439e3de68fc297b38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:03.581013 23056 tablet_service.cc:1467] Processing CreateTablet for tablet dedc2c023566419baac83160511d3efb (DEFAULT_TABLE table=test-table [id=7b8c1a85ef3143439e3de68fc297b38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:03.581465 23205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedc2c023566419baac83160511d3efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:03.581465 23056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedc2c023566419baac83160511d3efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:03.587241 23313 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Bootstrap starting.
I20260810 21:00:03.587839 22908 tablet_service.cc:1467] Processing CreateTablet for tablet dedc2c023566419baac83160511d3efb (DEFAULT_TABLE table=test-table [id=7b8c1a85ef3143439e3de68fc297b38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:03.588465 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedc2c023566419baac83160511d3efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:03.588984 23314 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Bootstrap starting.
I20260810 21:00:03.590400 23314 tablet_bootstrap.cc:654] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:03.591441 23314 log.cc:824] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:03.588760 23313 tablet_bootstrap.cc:654] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:03.592446 23314 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: No bootstrap required, opened a new log
I20260810 21:00:03.592531 23314 ts_tablet_manager.cc:1397] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 21:00:03.592650 23313 log.cc:824] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:03.593772 23313 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: No bootstrap required, opened a new log
I20260810 21:00:03.593886 23313 ts_tablet_manager.cc:1397] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260810 21:00:03.594936 23317 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Bootstrap starting.
I20260810 21:00:03.596541 23317 tablet_bootstrap.cc:654] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:03.597633 23317 log.cc:824] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:03.597618 23314 raft_consensus.cc:348] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.597628 23313 raft_consensus.cc:348] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.597802 23313 raft_consensus.cc:374] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:03.597793 23314 raft_consensus.cc:374] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:03.597844 23313 raft_consensus.cc:729] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89564e8aaf4f48f49a2b09b76363d9ca, State: Initialized, Role: FOLLOWER
I20260810 21:00:03.597860 23314 raft_consensus.cc:729] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ffe0f6df39453ea94a10b8630e582c, State: Initialized, Role: FOLLOWER
I20260810 21:00:03.597990 23314 consensus_queue.cc:260] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.597973 23313 consensus_queue.cc:260] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.598353 23313 ts_tablet_manager.cc:1428] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260810 21:00:03.598374 23314 ts_tablet_manager.cc:1428] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260810 21:00:03.598623 23317 tablet_bootstrap.cc:492] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: No bootstrap required, opened a new log
I20260810 21:00:03.598667 23317 ts_tablet_manager.cc:1397] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 21:00:03.598716 23129 heartbeater.cc:503] Master 127.14.200.62:39009 was elected leader, sending a full tablet report...
I20260810 21:00:03.598816 23281 heartbeater.cc:503] Master 127.14.200.62:39009 was elected leader, sending a full tablet report...
I20260810 21:00:03.602301 23317 raft_consensus.cc:348] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.602419 23317 raft_consensus.cc:374] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:03.602438 23317 raft_consensus.cc:729] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ee17ce95c946ebbce389522fe34c40, State: Initialized, Role: FOLLOWER
I20260810 21:00:03.602509 23317 consensus_queue.cc:260] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.602756 23317 ts_tablet_manager.cc:1428] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260810 21:00:03.602883 22980 heartbeater.cc:503] Master 127.14.200.62:39009 was elected leader, sending a full tablet report...
W20260810 21:00:03.638609 23132 tablet.cc:2369] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:00:03.735888 22982 tablet.cc:2369] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:03.761727 23319 raft_consensus.cc:482] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:03.761874 23319 raft_consensus.cc:504] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.762319 23319 leader_election.cc:288] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259), d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:03.766577 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.762638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:37551 (local address 127.14.200.2:43259)
0810 21:00:03.762686 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.762689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.762696 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:03.762725 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.762726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.762747 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.762807 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.763177 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.763981 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.765519 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.766191 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.766254 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.766261 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.766467 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.766509 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:00:03.766577 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.762678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:56159 (local address 127.14.200.1:42973)
0810 21:00:03.762729 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.762743 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.762751 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:03.762777 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.762778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.762787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.762839 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:03.763176 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.763981 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.765519 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.766131 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.766214 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.766225 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.766461 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.766505 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:00:03.766634 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.762571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42973 (local address 127.14.200.3:56159)
0810 21:00:03.762619 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.762630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.762676 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.762852 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.762853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.762854 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.763060 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.763062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.764076 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.764077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.765414 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.765417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.766209 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.766210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.766376 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.766379 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.766441 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.766497 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":56}
I20260810 21:00:03.766680 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.762492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43259 (local address 127.14.200.3:37551)
0810 21:00:03.762557 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:03.762572 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:03.762631 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:03.762841 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:03.762842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:03.762844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:03.762985 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.762987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.764076 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.764077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.765414 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:03.765417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.766257 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.766257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:03.766375 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.766379 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:03.766441 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 21:00:03.766497 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":29}
I20260810 21:00:03.767109 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedc2c023566419baac83160511d3efb" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ee17ce95c946ebbce389522fe34c40" is_pre_election: true
I20260810 21:00:03.767190 23077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedc2c023566419baac83160511d3efb" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ffe0f6df39453ea94a10b8630e582c" is_pre_election: true
I20260810 21:00:03.767299 22928 raft_consensus.cc:2454] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 0.
I20260810 21:00:03.767309 23077 raft_consensus.cc:2454] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 0.
I20260810 21:00:03.767555 23157 leader_election.cc:302] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08ffe0f6df39453ea94a10b8630e582c, 89564e8aaf4f48f49a2b09b76363d9ca; no voters:
I20260810 21:00:03.767745 23319 raft_consensus.cc:2781] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:03.767783 23319 raft_consensus.cc:482] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:03.767798 23319 raft_consensus.cc:3037] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:03.768867 23319 raft_consensus.cc:504] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.769021 23319 leader_election.cc:288] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 election: Requested vote from peers 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259), d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:03.769181 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedc2c023566419baac83160511d3efb" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ee17ce95c946ebbce389522fe34c40"
I20260810 21:00:03.769171 23077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedc2c023566419baac83160511d3efb" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ffe0f6df39453ea94a10b8630e582c"
I20260810 21:00:03.769260 22928 raft_consensus.cc:3037] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:03.769260 23077 raft_consensus.cc:3037] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:03.770771 22928 raft_consensus.cc:2454] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 1.
I20260810 21:00:03.770900 23077 raft_consensus.cc:2454] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 1.
I20260810 21:00:03.770961 23154 leader_election.cc:302] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89564e8aaf4f48f49a2b09b76363d9ca, d9ee17ce95c946ebbce389522fe34c40; no voters:
I20260810 21:00:03.771071 23319 raft_consensus.cc:2781] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:03.771245 23319 raft_consensus.cc:686] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 LEADER]: Becoming Leader. State: Replica: 89564e8aaf4f48f49a2b09b76363d9ca, State: Running, Role: LEADER
I20260810 21:00:03.771358 23319 consensus_queue.cc:237] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:03.772370 21923 catalog_manager.cc:5495] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3). New cstate: current_term: 1 leader_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } health_report { overall_health: UNKNOWN } } }
W20260810 21:00:03.792564 23284 tablet.cc:2369] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:03.810920 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.808397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.14.200.3:42721)
0810 21:00:03.808462 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.808465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.808474 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:03.808524 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.808525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.808546 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.808609 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:03.808736 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.809340 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.810072 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.810274 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.810325 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.810479 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:03.810586 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:03.810651 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.810684 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.810705 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 21:00:03.813608 22928 raft_consensus.cc:1261] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Refusing update from remote peer 89564e8aaf4f48f49a2b09b76363d9ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:03.813632 23077 raft_consensus.cc:1261] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Refusing update from remote peer 89564e8aaf4f48f49a2b09b76363d9ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:03.814136 23324 consensus_queue.cc:1035] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:03.814340 23319 consensus_queue.cc:1035] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:03.818509 23329 mvcc.cc:205] Tried to move back new op lower bound from 7317076393217331200 to 7317076393048231936. Current Snapshot: MvccSnapshot[applied={T|T < 7317076393217331200}]
I20260810 21:00:03.819610 23328 mvcc.cc:205] Tried to move back new op lower bound from 7317076393217331200 to 7317076393048231936. Current Snapshot: MvccSnapshot[applied={T|T < 7317076393217331200}]
I20260810 21:00:03.820283 21925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:03.820421 21925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:03.822047 21925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c31bf9e284f94a21866c0aaa6b293f14: Sending DeleteTablet for 3 replicas of tablet dedc2c023566419baac83160511d3efb
I20260810 21:00:03.822506 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet dedc2c023566419baac83160511d3efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:03 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:37544
I20260810 21:00:03.822592 23056 tablet_service.cc:1514] Processing DeleteTablet for tablet dedc2c023566419baac83160511d3efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:03 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:56248
I20260810 21:00:03.822889 23205 tablet_service.cc:1514] Processing DeleteTablet for tablet dedc2c023566419baac83160511d3efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:03 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:50594
I20260810 21:00:03.823122 23337 tablet_replica.cc:330] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: stopping tablet replica
I20260810 21:00:03.823339 23338 tablet_replica.cc:330] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: stopping tablet replica
I20260810 21:00:03.823361 23337 raft_consensus.cc:2229] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:03.823439 23338 raft_consensus.cc:2229] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:03.823535 23338 raft_consensus.cc:2258] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:03.823655 23337 raft_consensus.cc:2258] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:03.824225 23337 ts_tablet_manager.cc:1905] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:03.824708 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 21888
I20260810 21:00:03.824954 23338 ts_tablet_manager.cc:1905] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:03.827390 23337 ts_tablet_manager.cc:1918] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:03.827625 23338 ts_tablet_manager.cc:1918] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:03.827708 23338 log.cc:1196] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/dedc2c023566419baac83160511d3efb
I20260810 21:00:03.827670 23337 log.cc:1196] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/dedc2c023566419baac83160511d3efb
I20260810 21:00:03.828081 23338 ts_tablet_manager.cc:1939] T dedc2c023566419baac83160511d3efb P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting consensus metadata
I20260810 21:00:03.828300 23337 ts_tablet_manager.cc:1939] T dedc2c023566419baac83160511d3efb P 08ffe0f6df39453ea94a10b8630e582c: Deleting consensus metadata
I20260810 21:00:03.828593 23336 tablet_replica.cc:330] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: stopping tablet replica
I20260810 21:00:03.828888 23336 raft_consensus.cc:2229] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:03.829063 23336 raft_consensus.cc:2258] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:03.829593 23336 ts_tablet_manager.cc:1905] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:03.831427 23336 ts_tablet_manager.cc:1918] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:03.831537 23336 log.cc:1196] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/dedc2c023566419baac83160511d3efb
I20260810 21:00:03.831902 23336 ts_tablet_manager.cc:1939] T dedc2c023566419baac83160511d3efb P d9ee17ce95c946ebbce389522fe34c40: Deleting consensus metadata
W20260810 21:00:03.835392 22857 connection.cc:535] server connection from 127.0.0.1:37544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 21:00:03.835409 23157 connection.cc:535] server connection from 127.0.0.1:50594 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 21:00:03.835444 23007 connection.cc:535] server connection from 127.0.0.1:56248 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:00:03.845412 22983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.824173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.14.200.60:42683)
0810 21:00:03.824258 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.824260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.824270 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:03.824293 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.824293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.824306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.824356 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.837126 (+ 12770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.837924 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.839079 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.839389 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.839471 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.841083 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.841084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.841084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.841091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.844061 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.844488 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.844489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.844490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.844545 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.844760 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.844760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.844761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.844937 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.845016 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.845232 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.845310 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 21:00:03.845849 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.837801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.14.200.61:38445)
0810 21:00:03.837880 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.837881 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.837900 (+ 19us) server_negotiation.cc:407] Connection header received
0810 21:00:03.837922 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.837922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.837934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.838000 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.839325 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.840091 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.841162 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.841475 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.841518 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.842959 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.842961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.842961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.842969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.844580 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.845013 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.845014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.845015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.845073 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.845268 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.845269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.845269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.845392 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.845459 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.845660 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.845747 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 21:00:03.846052 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:03.837801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60248 (local address 127.14.200.60:42683)
0810 21:00:03.837847 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:03.837849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:03.838146 (+ 297us) server_negotiation.cc:407] Connection header received
0810 21:00:03.838244 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:03.838245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:03.838261 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:03.838293 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:03.839526 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.840252 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.841348 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:03.841669 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:03.841717 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:03.843228 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:03.843229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:03.843230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:03.843237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:03.844842 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.845205 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.845206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.845206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.845260 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:03.845477 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:03.845478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:03.845478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:03.845606 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:03.845665 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:03.845873 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 21:00:03.845960 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 21:00:04.818727 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:04.818336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:42305)
0810 21:00:04.818549 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:04.818650 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":67,"threads_started":1}
W20260810 21:00:04.819064 23129 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:39009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:04.820876 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:04.820565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:58729)
0810 21:00:04.820722 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:04.820830 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
W20260810 21:00:04.821072 22980 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:39009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:04.821148 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:04.820877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:37343)
0810 21:00:04.821031 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:04.821102 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
W20260810 21:00:04.821342 23281 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:39009 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:05.322522 23354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c31bf9e284f94a21866c0aaa6b293f14)
I20260810 21:00:05.322525 23353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c31bf9e284f94a21866c0aaa6b293f14)
I20260810 21:00:05.322641 23353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } }
I20260810 21:00:05.322715 23354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } }
I20260810 21:00:05.322904 23353 leader_election.cc:288] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009), 101f8d8de7db4b6da3f9366c4c20dbdb (127.14.200.61:38445)
I20260810 21:00:05.323057 23354 leader_election.cc:288] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009), 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60:42683)
I20260810 21:00:05.323295 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38580)
0810 21:00:05.323174 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.323246 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
W20260810 21:00:05.323477 22770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:05.323662 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38596)
0810 21:00:05.323449 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.323617 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":64,"threads_started":1}
W20260810 21:00:05.323776 22565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:05.324208 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.324118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38598)
0810 21:00:05.324155 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.324180 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 21:00:05.324244 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.324156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38606)
0810 21:00:05.324197 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.324217 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 21:00:05.324321 22770 leader_election.cc:334] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009): Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
W20260810 21:00:05.324303 22565 leader_election.cc:334] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009): Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:05.326560 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.14.200.60:42683)
0810 21:00:05.323709 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.323711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.323719 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:05.323745 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.323746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.323755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.323813 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:05.324054 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.324725 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.325838 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.326204 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.326286 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.326297 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.326458 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.326498 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":58,"threads_started":1}
I20260810 21:00:05.326563 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42683 (local address 127.0.0.1:60264)
0810 21:00:05.323537 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.323548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:05.323605 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:05.323867 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:05.323867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:05.323870 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:05.323927 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.323929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.324834 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.324835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.325748 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.325750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.326287 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.326288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.326354 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.326359 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:00:05.326449 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 21:00:05.326496 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":34,"threads_started":1}
I20260810 21:00:05.326771 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" is_pre_election: true
I20260810 21:00:05.326860 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.14.200.61:38445)
0810 21:00:05.323661 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.323663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.323674 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:00:05.323705 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.323706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.323718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.323797 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:05.324010 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.325143 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.326080 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.326455 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.326542 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.326552 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.326745 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.326789 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":37,"threads_started":1}
I20260810 21:00:05.326875 23356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.323099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38445 (local address 127.0.0.1:40322)
0810 21:00:05.323212 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.323223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:05.323285 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:05.323859 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:05.323860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:05.323863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:05.323925 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.323928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.325222 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.325223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.325978 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.325980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.326546 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.326547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.326631 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.326635 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:05.326692 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:00:05.326748 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260810 21:00:05.326902 22795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 101f8d8de7db4b6da3f9366c4c20dbdb in term 1.
I20260810 21:00:05.327083 22563 leader_election.cc:302] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101f8d8de7db4b6da3f9366c4c20dbdb, 9773b1ee71d242ad9dcb5629f5ad6d9e; no voters: c31bf9e284f94a21866c0aaa6b293f14
I20260810 21:00:05.327219 23354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 21:00:05.327265 23354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Starting leader election (detected failure of leader c31bf9e284f94a21866c0aaa6b293f14)
I20260810 21:00:05.327292 23354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:00:05.327316 22591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" is_pre_election: true
I20260810 21:00:05.327970 23354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } }
I20260810 21:00:05.328107 22591 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9773b1ee71d242ad9dcb5629f5ad6d9e in current term 2: Already voted for candidate 101f8d8de7db4b6da3f9366c4c20dbdb in this term.
I20260810 21:00:05.328135 23354 leader_election.cc:288] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 election: Requested vote from peers c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009), 9773b1ee71d242ad9dcb5629f5ad6d9e (127.14.200.60:42683)
I20260810 21:00:05.328287 22768 leader_election.cc:302] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9773b1ee71d242ad9dcb5629f5ad6d9e; no voters: 101f8d8de7db4b6da3f9366c4c20dbdb, c31bf9e284f94a21866c0aaa6b293f14
I20260810 21:00:05.328292 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.328210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38622)
0810 21:00:05.328241 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.328268 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 21:00:05.328266 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e"
I20260810 21:00:05.328354 22795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:00:05.328837 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.328702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38632)
0810 21:00:05.328748 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.328773 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 21:00:05.328922 22565 leader_election.cc:334] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009): Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
I20260810 21:00:05.329224 22795 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 101f8d8de7db4b6da3f9366c4c20dbdb in term 2.
I20260810 21:00:05.329315 23353 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:00:05.329406 22563 leader_election.cc:302] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 101f8d8de7db4b6da3f9366c4c20dbdb, 9773b1ee71d242ad9dcb5629f5ad6d9e; no voters: c31bf9e284f94a21866c0aaa6b293f14
I20260810 21:00:05.329555 23354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 2 FOLLOWER]: Leader election won for term 2
I20260810 21:00:05.329726 23354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [term 2 LEADER]: Becoming Leader. State: Replica: 101f8d8de7db4b6da3f9366c4c20dbdb, State: Running, Role: LEADER
I20260810 21:00:05.329820 23354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } }
I20260810 21:00:05.330184 23363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 101f8d8de7db4b6da3f9366c4c20dbdb. Latest consensus state: current_term: 2 leader_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:05.330263 23363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:05.330435 23366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:05.330916 23366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:05.331028 23366 catalog_manager.cc:1259] Loaded cluster ID: 87e6f592ab354a309a2ebb8bb9f0c966
I20260810 21:00:05.331063 23366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:00:05.331240 23366 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:05.331336 23366 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Loaded TSK: 0
I20260810 21:00:05.331475 23366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:00:05.379740 22581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:00:05.381529 22581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:05.382596 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.382476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38648)
0810 21:00:05.382533 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.382569 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 21:00:05.382624 22795 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [term 2 FOLLOWER]: Refusing update from remote peer 101f8d8de7db4b6da3f9366c4c20dbdb: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 21:00:05.382985 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.382867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.0.0.1:38650)
0810 21:00:05.382902 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.382937 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 21:00:05.382920 23363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 21:00:05.383097 22565 consensus_peers.cc:487] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb -> Peer c31bf9e284f94a21866c0aaa6b293f14 (127.14.200.62:39009): Couldn't send request to peer c31bf9e284f94a21866c0aaa6b293f14. Status: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 21:00:05.383613 23353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 101f8d8de7db4b6da3f9366c4c20dbdb. Latest consensus state: current_term: 2 leader_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:05.383728 23353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:00:05.384037 23368 mvcc.cc:205] Tried to move back new op lower bound from 7317076399645200384 to 7317076399431684096. Current Snapshot: MvccSnapshot[applied={T|T < 7317076399645200384}]
I20260810 21:00:05.384127 23354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:05.384229 23354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:05.384248 23369 mvcc.cc:205] Tried to move back new op lower bound from 7317076399645200384 to 7317076399431684096. Current Snapshot: MvccSnapshot[applied={T|T < 7317076399645200384}]
I20260810 21:00:05.384378 23353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c31bf9e284f94a21866c0aaa6b293f14" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39009 } } peers { permanent_uuid: "101f8d8de7db4b6da3f9366c4c20dbdb" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38445 } attrs { promote: false } } peers { permanent_uuid: "9773b1ee71d242ad9dcb5629f5ad6d9e" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42683 } attrs { promote: false } } }
I20260810 21:00:05.384471 23353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9773b1ee71d242ad9dcb5629f5ad6d9e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:00:05.389770 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:42721 (local address 127.0.0.1:50612)
0810 21:00:05.386509 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.386521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:05.386575 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:05.386912 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:05.386913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:05.386914 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:05.386946 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.386947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387883 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.387884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.388955 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.388958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.389537 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.389603 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.389606 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:05.389657 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 21:00:05.389698 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":51}
I20260810 21:00:05.390007 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.14.200.3:42721)
0810 21:00:05.386773 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.386775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.386783 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:05.386811 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.386812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.386824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.387012 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:05.387054 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387776 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389048 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.389439 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389534 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.389549 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.389880 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.389936 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260810 21:00:05.390219 23205 tablet_service.cc:1467] Processing CreateTablet for tablet 37683544ef6d4220a865fb7aa7959f51 (DEFAULT_TABLE table=test-table [id=05f788b708434decbba9c191062287ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:05.390323 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.14.200.1:42973)
0810 21:00:05.386954 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.386956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.386964 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:05.386992 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.386993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.387003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.387064 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:05.387246 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387726 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389115 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.389705 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389770 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.389785 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.390195 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.390248 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":58,"threads_started":1}
I20260810 21:00:05.390297 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42973 (local address 127.0.0.1:37550)
0810 21:00:05.386723 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.386733 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:05.386775 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:05.387113 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:05.387114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:05.387116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:05.387169 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.387171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387813 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.387814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.389013 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.389016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.389817 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.390019 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.390023 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:05.390180 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 21:00:05.390225 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":86,"threads_started":1}
I20260810 21:00:05.390380 23205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37683544ef6d4220a865fb7aa7959f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:05.390751 22908 tablet_service.cc:1467] Processing CreateTablet for tablet 37683544ef6d4220a865fb7aa7959f51 (DEFAULT_TABLE table=test-table [id=05f788b708434decbba9c191062287ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:05.390880 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37683544ef6d4220a865fb7aa7959f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:05.390972 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43259 (local address 127.0.0.1:56262)
0810 21:00:05.386425 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.386435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:05.386476 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:05.386854 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:05.386855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:05.386857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:05.386915 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.386917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387879 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.387881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.389583 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:05.389586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.390696 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.390698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:05.390786 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.390795 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 21:00:05.390863 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 21:00:05.390902 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":630}
I20260810 21:00:05.391245 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.386479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.14.200.2:43259)
0810 21:00:05.386703 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.386705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.386714 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:05.386743 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.386743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.386756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.386850 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:05.387012 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.387775 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.389652 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.390623 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.390698 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.390711 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.391135 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.391184 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":70,"threads_started":1}
I20260810 21:00:05.391453 23056 tablet_service.cc:1467] Processing CreateTablet for tablet 37683544ef6d4220a865fb7aa7959f51 (DEFAULT_TABLE table=test-table [id=05f788b708434decbba9c191062287ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:05.391592 23056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37683544ef6d4220a865fb7aa7959f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:05.392426 23379 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: Bootstrap starting.
I20260810 21:00:05.392318 23378 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Bootstrap starting.
I20260810 21:00:05.392839 23378 tablet_bootstrap.cc:654] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:05.393187 23381 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Bootstrap starting.
I20260810 21:00:05.393356 23379 tablet_bootstrap.cc:654] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:05.393469 23378 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: No bootstrap required, opened a new log
I20260810 21:00:05.393522 23378 ts_tablet_manager.cc:1397] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:05.393635 23378 raft_consensus.cc:348] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.393718 23378 raft_consensus.cc:374] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:05.393752 23378 raft_consensus.cc:729] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89564e8aaf4f48f49a2b09b76363d9ca, State: Initialized, Role: FOLLOWER
I20260810 21:00:05.393806 23378 consensus_queue.cc:260] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.393951 23378 ts_tablet_manager.cc:1428] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:05.394080 23379 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: No bootstrap required, opened a new log
I20260810 21:00:05.394135 23379 ts_tablet_manager.cc:1397] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:05.394299 23379 raft_consensus.cc:348] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.394351 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.394129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:50599)
0810 21:00:05.394276 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.394320 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260810 21:00:05.394382 23379 raft_consensus.cc:374] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:05.394407 23379 raft_consensus.cc:729] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ee17ce95c946ebbce389522fe34c40, State: Initialized, Role: FOLLOWER
I20260810 21:00:05.394474 23379 consensus_queue.cc:260] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.394639 23379 ts_tablet_manager.cc:1428] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:05.394750 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.394580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:51629)
0810 21:00:05.394700 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.394727 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260810 21:00:05.394819 23282 heartbeater.cc:503] Master 127.14.200.61:38445 was elected leader, sending a full tablet report...
I20260810 21:00:05.395490 22981 heartbeater.cc:503] Master 127.14.200.61:38445 was elected leader, sending a full tablet report...
I20260810 21:00:05.396039 23381 tablet_bootstrap.cc:654] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:05.396384 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.395005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:58027)
0810 21:00:05.396326 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.396358 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":79,"threads_started":1}
I20260810 21:00:05.396643 23381 tablet_bootstrap.cc:492] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: No bootstrap required, opened a new log
I20260810 21:00:05.396710 23381 ts_tablet_manager.cc:1397] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 21:00:05.396901 23381 raft_consensus.cc:348] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.396978 23381 raft_consensus.cc:374] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:05.397002 23381 raft_consensus.cc:729] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ffe0f6df39453ea94a10b8630e582c, State: Initialized, Role: FOLLOWER
I20260810 21:00:05.397063 23381 consensus_queue.cc:260] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.397222 23381 ts_tablet_manager.cc:1428] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:05.397359 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.396666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:52491)
0810 21:00:05.397299 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.397332 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616}
I20260810 21:00:05.397573 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.397355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:54817)
0810 21:00:05.397501 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.397542 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260810 21:00:05.397728 23131 heartbeater.cc:503] Master 127.14.200.61:38445 was elected leader, sending a full tablet report...
I20260810 21:00:05.397989 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.397798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:42639)
0810 21:00:05.397916 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.397952 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":79,"threads_started":1}
I20260810 21:00:05.512435 23383 raft_consensus.cc:482] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:05.512554 23383 raft_consensus.cc:504] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.512737 23383 leader_election.cc:288] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259), d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:05.513062 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37683544ef6d4220a865fb7aa7959f51" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ee17ce95c946ebbce389522fe34c40" is_pre_election: true
I20260810 21:00:05.513049 23077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37683544ef6d4220a865fb7aa7959f51" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ffe0f6df39453ea94a10b8630e582c" is_pre_election: true
I20260810 21:00:05.513171 22928 raft_consensus.cc:2454] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 0.
I20260810 21:00:05.513176 23077 raft_consensus.cc:2454] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 0.
I20260810 21:00:05.513361 23157 leader_election.cc:302] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08ffe0f6df39453ea94a10b8630e582c, 89564e8aaf4f48f49a2b09b76363d9ca; no voters:
I20260810 21:00:05.513484 23383 raft_consensus.cc:2781] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:05.513525 23383 raft_consensus.cc:482] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:05.513549 23383 raft_consensus.cc:3037] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:05.514353 23383 raft_consensus.cc:504] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.514519 23383 leader_election.cc:288] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 election: Requested vote from peers 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259), d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973)
I20260810 21:00:05.514703 23077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37683544ef6d4220a865fb7aa7959f51" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08ffe0f6df39453ea94a10b8630e582c"
I20260810 21:00:05.514794 23077 raft_consensus.cc:3037] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:05.514740 22928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37683544ef6d4220a865fb7aa7959f51" candidate_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ee17ce95c946ebbce389522fe34c40"
I20260810 21:00:05.514837 22928 raft_consensus.cc:3037] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:05.515453 22928 raft_consensus.cc:2454] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 1.
I20260810 21:00:05.515462 23077 raft_consensus.cc:2454] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89564e8aaf4f48f49a2b09b76363d9ca in term 1.
I20260810 21:00:05.515676 23154 leader_election.cc:302] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89564e8aaf4f48f49a2b09b76363d9ca, d9ee17ce95c946ebbce389522fe34c40; no voters:
I20260810 21:00:05.515805 23383 raft_consensus.cc:2781] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:05.516024 23383 raft_consensus.cc:686] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 LEADER]: Becoming Leader. State: Replica: 89564e8aaf4f48f49a2b09b76363d9ca, State: Running, Role: LEADER
I20260810 21:00:05.516125 23383 consensus_queue.cc:237] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } }
I20260810 21:00:05.516381 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.516274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:59829)
0810 21:00:05.516312 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.516344 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260810 21:00:05.516861 22581 catalog_manager.cc:5495] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3). New cstate: current_term: 1 leader_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89564e8aaf4f48f49a2b09b76363d9ca" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 42721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 } health_report { overall_health: UNKNOWN } } }
I20260810 21:00:05.536041 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.533843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.14.200.3:42721)
0810 21:00:05.533905 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:05.533911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:05.533918 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:05.533944 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:05.533945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:05.533955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:05.534006 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:05.534128 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.534654 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.535383 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:05.535598 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:05.535664 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:05.535778 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:05.535804 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:05.535841 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:05.535914 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 21:00:05.535935 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:00:05.537045 23077 raft_consensus.cc:1261] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Refusing update from remote peer 89564e8aaf4f48f49a2b09b76363d9ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:05.537101 22928 raft_consensus.cc:1261] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Refusing update from remote peer 89564e8aaf4f48f49a2b09b76363d9ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:05.537292 23383 consensus_queue.cc:1035] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "08ffe0f6df39453ea94a10b8630e582c" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:05.537401 23392 consensus_queue.cc:1035] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ee17ce95c946ebbce389522fe34c40" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:05.538280 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.538142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:35627)
0810 21:00:05.538203 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.538244 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:00:05.538404 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.537993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:33297)
0810 21:00:05.538339 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.538374 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323}
I20260810 21:00:05.539533 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.539400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:37671)
0810 21:00:05.539469 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.539495 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 21:00:05.539956 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.539842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.1:34807)
0810 21:00:05.539898 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.539929 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 21:00:05.540383 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.540261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.2:51595)
0810 21:00:05.540320 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.540352 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 21:00:05.540596 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.540503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39009 (local address 127.14.200.3:59093)
0810 21:00:05.540541 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.540570 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 21:00:05.541417 22579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:05.541510 22579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:05.542882 22579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 101f8d8de7db4b6da3f9366c4c20dbdb: Sending DeleteTablet for 3 replicas of tablet 37683544ef6d4220a865fb7aa7959f51
I20260810 21:00:05.543216 23205 tablet_service.cc:1514] Processing DeleteTablet for tablet 37683544ef6d4220a865fb7aa7959f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:05 UTC) from {username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:50612
I20260810 21:00:05.543300 23056 tablet_service.cc:1514] Processing DeleteTablet for tablet 37683544ef6d4220a865fb7aa7959f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:05 UTC) from {username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:56262
I20260810 21:00:05.543339 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet 37683544ef6d4220a865fb7aa7959f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:05 UTC) from {username='oryx', principal='oryx/127.14.200.61@KRBTEST.COM'} at 127.0.0.1:37550
I20260810 21:00:05.543478 23405 tablet_replica.cc:330] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: stopping tablet replica
I20260810 21:00:05.543531 23405 raft_consensus.cc:2229] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:05.543567 23405 raft_consensus.cc:2258] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:05.543555 23406 tablet_replica.cc:330] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40: stopping tablet replica
I20260810 21:00:05.543603 23406 raft_consensus.cc:2229] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:05.543661 23406 raft_consensus.cc:2258] T 37683544ef6d4220a865fb7aa7959f51 P d9ee17ce95c946ebbce389522fe34c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:05.543783 23404 tablet_replica.cc:330] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: stopping tablet replica
I20260810 21:00:05.543882 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22744
I20260810 21:00:05.544207 23404 raft_consensus.cc:2229] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:05.544282 23404 raft_consensus.cc:2258] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:05.549973 23404 ts_tablet_manager.cc:1905] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:05.551522 23404 ts_tablet_manager.cc:1918] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:05.551520 23405 ts_tablet_manager.cc:1905] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:05.551589 23404 log.cc:1196] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/37683544ef6d4220a865fb7aa7959f51
I20260810 21:00:05.551847 23404 ts_tablet_manager.cc:1939] T 37683544ef6d4220a865fb7aa7959f51 P 89564e8aaf4f48f49a2b09b76363d9ca: Deleting consensus metadata
I20260810 21:00:05.552348 22563 catalog_manager.cc:4841] TS 89564e8aaf4f48f49a2b09b76363d9ca (127.14.200.3:42721): tablet 37683544ef6d4220a865fb7aa7959f51 (table test-table [id=05f788b708434decbba9c191062287ec]) successfully deleted
I20260810 21:00:05.552996 23405 ts_tablet_manager.cc:1918] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:05.553056 23405 log.cc:1196] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/37683544ef6d4220a865fb7aa7959f51
I20260810 21:00:05.553280 23405 ts_tablet_manager.cc:1939] T 37683544ef6d4220a865fb7aa7959f51 P 08ffe0f6df39453ea94a10b8630e582c: Deleting consensus metadata
I20260810 21:00:05.553702 22565 catalog_manager.cc:4841] TS 08ffe0f6df39453ea94a10b8630e582c (127.14.200.2:43259): tablet 37683544ef6d4220a865fb7aa7959f51 (table test-table [id=05f788b708434decbba9c191062287ec]) successfully deleted
I20260810 21:00:05.555037 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:05.554904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42973 (local address 127.0.0.1:37566)
0810 21:00:05.554964 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:05.554995 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 21:00:05.555147 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22989
W20260810 21:00:05.555202 22562 catalog_manager.cc:4584] TS d9ee17ce95c946ebbce389522fe34c40 (127.14.200.1:42973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37683544ef6d4220a865fb7aa7959f51: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42973: connect: Connection refused (error 111)
I20260810 21:00:05.563500 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 23137
I20260810 21:00:05.571686 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22431
I20260810 21:00:05.578418 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 22641
2026-08-10T21:00:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4969 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:05 dist-test-slave-1x4g krb5kdc[23409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:05 dist-test-slave-1x4g krb5kdc[23409](info): set up 2 sockets
Aug 10 21:00:05 dist-test-slave-1x4g krb5kdc[23409](info): commencing operation
krb5kdc: starting...
W20260810 21:00:06.666940 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:06 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab.
Entry for principal oryx/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab.
I20260810 21:00:06.789779 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41297
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab
--principal=oryx/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:06.874270 23422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:06.874485 23422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:06.874505 23422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:06.875653 23422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:06.875687 23422 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 21:00:06.875700 23422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:06.875713 23422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:06.875725 23422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:06.875737 23422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:06.877207 23422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41297
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.62.keytab
--principal=oryx/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:06.877408 23422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:06.877586 23422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:06.882305 23432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:06.882350 23430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:06.882552 23422 server_base.cc:1034] running on GCE node
W20260810 21:00:06.882305 23429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:06.882879 23422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:06.883118 23422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:06.884258 23422 hybrid_clock.cc:648] HybridClock initialized: now 1786395606884236 us; error 39 us; skew 500 ppm
Aug 10 21:00:06 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395606, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:06.886831 23422 init.cc:377] Logged in from keytab as oryx/127.14.200.62@KRBTEST.COM (short username oryx)
I20260810 21:00:06.887316 23422 webserver.cc:460] Webserver started at http://127.14.200.62:33673/ using document root <none> and password file <none>
I20260810 21:00:06.887503 23422 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:06.887543 23422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:06.887643 23422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:06.888669 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "fb79b5305af44341b9fc370292ff5c76"
format_stamp: "Formatted at 2026-08-10 21:00:06 on dist-test-slave-1x4g"
server_key: "2ae386a0acdf2ef86fcd24a5ee8e7e3a"
server_key_iv: "0f81ca350442795c9f87fe42d952f3c5"
server_key_version: "encryptionkey@0"
I20260810 21:00:06.889067 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "fb79b5305af44341b9fc370292ff5c76"
format_stamp: "Formatted at 2026-08-10 21:00:06 on dist-test-slave-1x4g"
server_key: "2ae386a0acdf2ef86fcd24a5ee8e7e3a"
server_key_iv: "0f81ca350442795c9f87fe42d952f3c5"
server_key_version: "encryptionkey@0"
I20260810 21:00:06.890367 23422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:06.891050 23440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:06.891215 23422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:06.891325 23422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "fb79b5305af44341b9fc370292ff5c76"
format_stamp: "Formatted at 2026-08-10 21:00:06 on dist-test-slave-1x4g"
server_key: "2ae386a0acdf2ef86fcd24a5ee8e7e3a"
server_key_iv: "0f81ca350442795c9f87fe42d952f3c5"
server_key_version: "encryptionkey@0"
I20260810 21:00:06.891402 23422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:06.912463 23422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:06.914615 23422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:06.914798 23422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:06.919947 23422 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41297
I20260810 21:00:06.920001 23509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41297 every 8 connection(s)
I20260810 21:00:06.920347 23422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:06.921041 23510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:06.923460 23510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Bootstrap starting.
I20260810 21:00:06.924118 23510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:06.924432 23510 log.cc:824] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:06.924892 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 23422
I20260810 21:00:06.925028 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:06.925256 15136 external_mini_cluster.cc:1428] Setting key 00c9ac8a86f504d245e70e8fc4a45410
I20260810 21:00:06.925346 23510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: No bootstrap required, opened a new log
I20260810 21:00:06.928320 23510 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } }
I20260810 21:00:06.928498 23510 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:06.928532 23510 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb79b5305af44341b9fc370292ff5c76, State: Initialized, Role: FOLLOWER
I20260810 21:00:06.928653 23510 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } }
I20260810 21:00:06.928725 23510 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:06.928753 23510 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:06.928817 23510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:06.929491 23510 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } }
I20260810 21:00:06.929589 23510 leader_election.cc:302] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb79b5305af44341b9fc370292ff5c76; no voters:
I20260810 21:00:06.929752 23510 leader_election.cc:288] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:06.929806 23515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:06.929979 23515 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [term 1 LEADER]: Becoming Leader. State: Replica: fb79b5305af44341b9fc370292ff5c76, State: Running, Role: LEADER
I20260810 21:00:06.930056 23510 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:06.930086 23515 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } }
I20260810 21:00:06.930421 23515 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb79b5305af44341b9fc370292ff5c76. Latest consensus state: current_term: 1 leader_uuid: "fb79b5305af44341b9fc370292ff5c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } } }
I20260810 21:00:06.930519 23515 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:06.930430 23517 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb79b5305af44341b9fc370292ff5c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb79b5305af44341b9fc370292ff5c76" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41297 } } }
I20260810 21:00:06.930763 23517 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:06.931082 23520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:06.931893 23520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 21:00:06 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:06.934828 23520 catalog_manager.cc:1347] Generated new cluster ID: ae2cd50a4b2d4a2a8779783e1d286b67
I20260810 21:00:06.934886 23520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:00:06.937623 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:06.925827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.14.200.62:41297)
0810 21:00:06.926045 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:06.926056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:06.926095 (+ 39us) server_negotiation.cc:407] Connection header received
0810 21:00:06.926852 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:06.926862 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:06.926940 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:06.927101 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:06.927703 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:06.929069 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:06.930093 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:06.930826 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:06.930946 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:06.934145 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:06.934148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:06.934149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:06.934161 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:06.935890 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:06.936303 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:06.936307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:06.936307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:06.936417 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:06.936655 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:06.936655 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:06.936656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:06.937001 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:06.937182 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:06.937284 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 21:00:06.937440 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260810 21:00:06.944010 23520 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:06.944723 23520 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:06.949568 23520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Generated new TSK 0
I20260810 21:00:06.949782 23520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
I20260810 21:00:06.975845 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41297
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:07.061023 23538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:07.061264 23538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:07.061302 23538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:07.062472 23538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:07.062537 23538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:07.062573 23538 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 21:00:07.062631 23538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:07.064172 23538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:07.064462 23538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:07.064710 23538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:00:07.070026 23538 server_base.cc:1034] running on GCE node
W20260810 21:00:07.070065 23544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:07.070031 23543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:07.070201 23546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:07.070434 23538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:07.070655 23538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:07.071798 23538 hybrid_clock.cc:648] HybridClock initialized: now 1786395607071788 us; error 30 us; skew 500 ppm
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:07.074433 23538 init.cc:377] Logged in from keytab as oryx/127.14.200.1@KRBTEST.COM (short username oryx)
I20260810 21:00:07.074890 23538 webserver.cc:460] Webserver started at http://127.14.200.1:39617/ using document root <none> and password file <none>
I20260810 21:00:07.075140 23538 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:07.075210 23538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:07.075369 23538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:07.076469 23538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "6e8d95c3d8604aecac833a354741032c"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "e970e43db99cce92062a587380d37734"
server_key_iv: "bc649255a7afb09b68201bfe6e04b069"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.076917 23538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "6e8d95c3d8604aecac833a354741032c"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "e970e43db99cce92062a587380d37734"
server_key_iv: "bc649255a7afb09b68201bfe6e04b069"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.078282 23538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:07.079087 23553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.079270 23538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:00:07.079375 23538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "6e8d95c3d8604aecac833a354741032c"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "e970e43db99cce92062a587380d37734"
server_key_iv: "bc649255a7afb09b68201bfe6e04b069"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.079464 23538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:07.107456 23538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:07.109525 23538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:07.109706 23538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:07.110270 23538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:07.110338 23538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.110387 23538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:07.110415 23538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.117897 23538 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:41493
I20260810 21:00:07.117982 23677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:41493 every 8 connection(s)
I20260810 21:00:07.118383 23538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:07.120968 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.110831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:35083 (local address 127.14.200.62:41297)
0810 21:00:07.110909 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.110913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.111143 (+ 230us) server_negotiation.cc:407] Connection header received
0810 21:00:07.111651 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.111652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.111665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.111757 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.112695 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.113567 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.114591 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.114961 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.115045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.117910 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.117912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.117912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.117921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.119476 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.119906 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.119908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.119908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.119981 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.120247 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.120248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.120248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.120373 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.120448 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.120734 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.120874 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 21:00:07.121115 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.110808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.1:35083)
0810 21:00:07.110982 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.111039 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.111544 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.111826 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.111830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.112284 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.112583 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.112590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.113676 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.113677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.114521 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.114523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.115089 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.115092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.115254 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.115795 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.115818 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.117809 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.119582 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.119590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.119592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.119825 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.120055 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.120056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.120057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.120176 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.120440 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.120447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.120595 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.120700 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.120895 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260810 21:00:07.121634 23678 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41297
I20260810 21:00:07.121759 23678 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:07.121944 23678 heartbeater.cc:511] Master 127.14.200.62:41297 requested a full tablet report, sending...
I20260810 21:00:07.122275 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 23538
I20260810 21:00:07.122402 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:07.122625 15136 external_mini_cluster.cc:1428] Setting key c35ace1793b6e4b82c007259aaf95d1e
I20260810 21:00:07.122635 23457 ts_manager.cc:194] Registered new tserver with Master: 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493)
I20260810 21:00:07.123528 23457 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:35083
I20260810 21:00:07.129320 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.122753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37787 (local address 127.14.200.62:41297)
0810 21:00:07.122800 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.122802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.123034 (+ 232us) server_negotiation.cc:407] Connection header received
0810 21:00:07.123105 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.123106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.123116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.123159 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.123505 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.124160 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.125290 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.125602 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.125674 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.126669 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.126670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.126671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.126678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.128142 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.128569 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.128570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.128570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.128621 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.128851 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.128851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.128852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.128959 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.129011 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.129198 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.129257 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:00:07.129453 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.122702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.1:37787)
0810 21:00:07.122975 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.122986 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.123050 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.123172 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.123173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.123342 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.123446 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.123448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.124279 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.124280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.125175 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.125179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.125681 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.125681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.125741 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.126209 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.126215 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.126584 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.128223 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.128224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.128225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.128474 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.128685 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.128686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.128686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.128764 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.129026 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.129027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.129095 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.129184 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.129306 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
I20260810 21:00:07.151275 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41297
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:07.236901 23687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:07.237133 23687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:07.237169 23687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:07.238344 23687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:07.238408 23687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:07.238442 23687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 21:00:07.238478 23687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:07.239967 23687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:07.240224 23687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:07.240449 23687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:07.245481 23692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:07.245628 23687 server_base.cc:1034] running on GCE node
W20260810 21:00:07.245491 23693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:07.245687 23695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:07.245962 23687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:07.246245 23687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:07.247422 23687 hybrid_clock.cc:648] HybridClock initialized: now 1786395607247408 us; error 40 us; skew 500 ppm
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:07.250118 23687 init.cc:377] Logged in from keytab as oryx/127.14.200.2@KRBTEST.COM (short username oryx)
I20260810 21:00:07.250548 23687 webserver.cc:460] Webserver started at http://127.14.200.2:44901/ using document root <none> and password file <none>
I20260810 21:00:07.250771 23687 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:07.250837 23687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:07.250996 23687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:07.252079 23687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "847cb1cc324f421e966123f6eb8a0ad0"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "a22f9aafa83730cd7454a67440455d38"
server_key_iv: "bb5c0561e085ac181fbef9bdcfb52dbb"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.252467 23687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "847cb1cc324f421e966123f6eb8a0ad0"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "a22f9aafa83730cd7454a67440455d38"
server_key_iv: "bb5c0561e085ac181fbef9bdcfb52dbb"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.253880 23687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:07.254616 23702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.254776 23687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:07.254880 23687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "847cb1cc324f421e966123f6eb8a0ad0"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "a22f9aafa83730cd7454a67440455d38"
server_key_iv: "bb5c0561e085ac181fbef9bdcfb52dbb"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.254982 23687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:07.267144 23687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:07.269698 23687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:07.269865 23687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:07.270403 23687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:07.270463 23687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.270505 23687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:07.270537 23687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.278223 23687 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:45565
I20260810 21:00:07.278302 23825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:45565 every 8 connection(s)
I20260810 21:00:07.278661 23687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:00:07.281133 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.270905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:51327 (local address 127.14.200.62:41297)
0810 21:00:07.270975 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.270977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.271247 (+ 270us) server_negotiation.cc:407] Connection header received
0810 21:00:07.271999 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.272000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.272010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.272064 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.273023 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.273862 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.275525 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.275785 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.275874 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.278164 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.278166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.278166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.278176 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.279726 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.280125 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.280127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.280127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.280187 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.280442 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.280442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.280443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.280558 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.280627 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.280957 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.281038 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:00:07.281322 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.270871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.2:51327)
0810 21:00:07.271095 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.271148 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.271891 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.272144 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.272149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.272576 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.272904 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.272912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.273968 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.273970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.275434 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.275437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.275888 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.275889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.276031 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.276526 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.276534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.278067 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.279818 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.279825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.279827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.280048 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.280254 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.280255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.280255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.280368 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.280631 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.280638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.280769 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.280935 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.281105 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260810 21:00:07.281842 23826 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41297
I20260810 21:00:07.281925 23826 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:07.282130 23826 heartbeater.cc:511] Master 127.14.200.62:41297 requested a full tablet report, sending...
I20260810 21:00:07.282572 23458 ts_manager.cc:194] Registered new tserver with Master: 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:07.283089 23458 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:51327
I20260810 21:00:07.287472 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 23687
I20260810 21:00:07.287596 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:07.287854 15136 external_mini_cluster.cc:1428] Setting key 8805b085821d1ae75e7e8c5e6a6f7712
I20260810 21:00:07.288208 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.282905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.2:56653)
0810 21:00:07.283038 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.283048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.283110 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.283262 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.283263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.283464 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.283543 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.283545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.284268 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.284268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.284742 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.284743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.285150 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.285151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.285204 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.285540 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.285544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.285806 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.287046 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.287047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.287048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.287201 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.287402 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.287403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.287404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.287477 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.287835 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.287836 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.287884 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.287989 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.288088 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260810 21:00:07.288211 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.282968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:56653 (local address 127.14.200.62:41297)
0810 21:00:07.283012 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.283014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.283110 (+ 96us) server_negotiation.cc:407] Connection header received
0810 21:00:07.283183 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.283184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.283196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.283251 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.283618 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.284181 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.284870 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.285079 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.285129 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.285879 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.285880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.285880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.285886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.286977 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.287287 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.287288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.287288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.287338 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.287554 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.287554 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.287554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.287752 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.287842 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.288037 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.288120 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for oryx/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
I20260810 21:00:07.314851 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41297
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:07.399083 23835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:07.399318 23835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:07.399358 23835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:07.400503 23835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:07.400569 23835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:07.400604 23835 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 21:00:07.400645 23835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:07.402129 23835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.200.62:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:07.402390 23835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:07.402621 23835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:00:07.407660 23835 server_base.cc:1034] running on GCE node
W20260810 21:00:07.407575 23844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:07.407661 23840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:07.407575 23842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:07.407964 23835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:07.408191 23835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:07.409327 23835 hybrid_clock.cc:648] HybridClock initialized: now 1786395607409316 us; error 31 us; skew 500 ppm
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:07.411829 23835 init.cc:377] Logged in from keytab as oryx/127.14.200.3@KRBTEST.COM (short username oryx)
I20260810 21:00:07.412206 23835 webserver.cc:460] Webserver started at http://127.14.200.3:39045/ using document root <none> and password file <none>
I20260810 21:00:07.412402 23835 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:07.412442 23835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:07.412539 23835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:07.413591 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "ba8aee1ea0ba44fd9e24d67424fcf208"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "624c461ac48e01200b0a198ea47f3b1b"
server_key_iv: "96a97fba238e5bcef3476a044de3ca66"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.413888 23835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "ba8aee1ea0ba44fd9e24d67424fcf208"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "624c461ac48e01200b0a198ea47f3b1b"
server_key_iv: "96a97fba238e5bcef3476a044de3ca66"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.415186 23835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:07.415894 23851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.416074 23835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:07.416153 23835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ba8aee1ea0ba44fd9e24d67424fcf208"
format_stamp: "Formatted at 2026-08-10 21:00:07 on dist-test-slave-1x4g"
server_key: "624c461ac48e01200b0a198ea47f3b1b"
server_key_iv: "96a97fba238e5bcef3476a044de3ca66"
server_key_version: "encryptionkey@0"
I20260810 21:00:07.416206 23835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:07.428337 23835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:07.430423 23835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:07.430583 23835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:07.431231 23835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:07.431270 23835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.431291 23835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:07.431324 23835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.438887 23835 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:37747
I20260810 21:00:07.439230 23977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:37747 every 8 connection(s)
I20260810 21:00:07.439355 23835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:07.440436 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 23835
I20260810 21:00:07.440554 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:07.440832 15136 external_mini_cluster.cc:1428] Setting key 48666c30eea42b0a212033a48e551131
I20260810 21:00:07.441845 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.431687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:37373 (local address 127.14.200.62:41297)
0810 21:00:07.431742 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.431745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.431930 (+ 185us) server_negotiation.cc:407] Connection header received
0810 21:00:07.432416 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.432417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.432440 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.432521 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.433323 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.434160 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.435211 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.435566 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.435657 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.438635 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.438637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.438638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.438647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.440207 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.440697 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.440698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.440699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.440760 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.441129 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.441130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.441130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.441262 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.441343 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.441649 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.441730 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:00:07.442001 23862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.431648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.3:37373)
0810 21:00:07.431801 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.431840 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.432334 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.432574 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.432577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.432906 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.433211 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.433218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.434294 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.434295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.435140 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.435142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.435696 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.435697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.435846 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.436434 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.436445 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.438505 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.440313 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.440321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.440323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.440598 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.440877 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.440878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.440878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.440992 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.441335 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.441342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.441507 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.441615 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.441773 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260810 21:00:07.442451 23978 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41297
I20260810 21:00:07.442543 23978 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:07.442739 23978 heartbeater.cc:511] Master 127.14.200.62:41297 requested a full tablet report, sending...
I20260810 21:00:07.443136 23457 ts_manager.cc:194] Registered new tserver with Master: ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747)
I20260810 21:00:07.443653 23457 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:37373
I20260810 21:00:07.448558 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.443534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:52997 (local address 127.14.200.62:41297)
0810 21:00:07.443609 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.443610 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.443722 (+ 112us) server_negotiation.cc:407] Connection header received
0810 21:00:07.443782 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.443783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.443792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.443851 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.444265 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.444706 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.445402 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.445579 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.445635 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.446379 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.446380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.446381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.446387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.447751 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.448022 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.448023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.448023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.448070 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.448212 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.448212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.448213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.448291 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.448337 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.448449 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.448501 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:00:07.448571 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.443454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.14.200.3:52997)
0810 21:00:07.443647 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.443657 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.443719 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.443857 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.443857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.444098 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.444189 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.444191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.444831 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.444832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.445333 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.445335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.445652 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.445652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.445702 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.446050 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.446053 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.446307 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.447818 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.447819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.447819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.447964 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.448126 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.448126 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.448126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.448175 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.448328 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.448328 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.448378 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.448423 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.448495 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260810 21:00:07.452383 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 21:00:07.474642 23514 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:07.469042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.14.200.62:41297)
0810 21:00:07.469105 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.469107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.469119 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:00:07.469191 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.469192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.469203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.469278 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.469873 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.470363 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.471119 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.471328 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.471405 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.474501 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51240: BlockingRecv error: recv got EOF from 127.0.0.1:51240 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.481755 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.475403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.14.200.62:41297)
0810 21:00:07.475445 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.475447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.475547 (+ 100us) server_negotiation.cc:407] Connection header received
0810 21:00:07.475568 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.475569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.475578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.475618 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.476046 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.476485 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.477188 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.477360 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.477412 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.479315 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.479316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.479316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.479322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.480669 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.481033 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.481034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.481034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.481087 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.481281 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.481282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.481282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.481384 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.481461 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.481556 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.481612 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:00:07.483392 23457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:00:07.484997 23457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:07.491384 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:37747 (local address 127.0.0.1:38096)
0810 21:00:07.487888 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.487898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.487954 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.488206 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.488207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.488209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.488252 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.488253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.489279 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.489280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.490166 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.490169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.491151 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.491152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.491213 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.491217 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.491262 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.491301 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260810 21:00:07.491395 24001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41493 (local address 127.0.0.1:41722)
0810 21:00:07.487436 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.487477 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.487543 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.487820 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.487823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.487826 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.487887 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.487888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.489003 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.489004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.490350 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.490353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.491055 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.491056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.491148 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.491163 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.491238 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.491297 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":47,"threads_started":1}
I20260810 21:00:07.491407 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.14.200.1:41493)
0810 21:00:07.487623 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.487629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.487641 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:00:07.487683 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.487685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.487699 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.487779 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.488045 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.488909 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.490469 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.490960 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.491054 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.491136 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.491255 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.491296 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
I20260810 21:00:07.491711 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.14.200.3:37747)
0810 21:00:07.487679 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.487684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.488018 (+ 334us) server_negotiation.cc:407] Connection header received
0810 21:00:07.488064 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.488066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.488082 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.488151 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.488313 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.489195 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.490230 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.490978 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.491046 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.491117 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.491575 (+ 458us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.491617 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
I20260810 21:00:07.492467 23608 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4e3c70611a4888ab675577f496f642 (DEFAULT_TABLE table=test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.492463 23906 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4e3c70611a4888ab675577f496f642 (DEFAULT_TABLE table=test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.492702 24002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.0.0.1:49098)
0810 21:00:07.488241 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.488251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.488306 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.489046 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.489047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.489050 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.489090 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.489092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.490441 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.490442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.491492 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.491496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.492475 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.492476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.492535 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.492540 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.492597 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.492636 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":63,"threads_started":1}
I20260810 21:00:07.492801 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.487701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.14.200.2:45565)
0810 21:00:07.487886 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.487891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.488842 (+ 951us) server_negotiation.cc:407] Connection header received
0810 21:00:07.488889 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.488892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.488907 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.488979 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.489168 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.490336 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.491585 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.492292 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.492360 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.492426 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.492630 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.492676 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260810 21:00:07.492887 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4e3c70611a4888ab675577f496f642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.493141 23906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4e3c70611a4888ab675577f496f642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.493358 23756 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4e3c70611a4888ab675577f496f642 (DEFAULT_TABLE table=test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.493681 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4e3c70611a4888ab675577f496f642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.497200 24008 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Bootstrap starting.
I20260810 21:00:07.497356 24007 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Bootstrap starting.
I20260810 21:00:07.497685 24010 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Bootstrap starting.
I20260810 21:00:07.497907 24008 tablet_bootstrap.cc:654] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.498037 24007 tablet_bootstrap.cc:654] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.498245 24008 log.cc:824] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:07.498327 24007 log.cc:824] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:07.498376 24010 tablet_bootstrap.cc:654] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.498703 24010 log.cc:824] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:07.499053 24007 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: No bootstrap required, opened a new log
I20260810 21:00:07.499133 24007 ts_tablet_manager.cc:1397] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:00:07.499418 24010 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: No bootstrap required, opened a new log
I20260810 21:00:07.499500 24010 ts_tablet_manager.cc:1397] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:00:07.501123 24008 tablet_bootstrap.cc:492] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: No bootstrap required, opened a new log
I20260810 21:00:07.501206 24008 ts_tablet_manager.cc:1397] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 21:00:07.501145 24007 raft_consensus.cc:348] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.501317 24007 raft_consensus.cc:374] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.501358 24007 raft_consensus.cc:729] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.501477 24007 consensus_queue.cc:260] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.501417 24010 raft_consensus.cc:348] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.501564 24010 raft_consensus.cc:374] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.501610 24010 raft_consensus.cc:729] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.501747 24007 ts_tablet_manager.cc:1428] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:00:07.501721 24010 consensus_queue.cc:260] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.501873 23678 heartbeater.cc:503] Master 127.14.200.62:41297 was elected leader, sending a full tablet report...
I20260810 21:00:07.501971 24010 ts_tablet_manager.cc:1428] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 21:00:07.502140 23826 heartbeater.cc:503] Master 127.14.200.62:41297 was elected leader, sending a full tablet report...
I20260810 21:00:07.503396 24008 raft_consensus.cc:348] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.503539 24008 raft_consensus.cc:374] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.503584 24008 raft_consensus.cc:729] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.503700 24008 consensus_queue.cc:260] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.503949 24008 ts_tablet_manager.cc:1428] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:00:07.504025 23978 heartbeater.cc:503] Master 127.14.200.62:41297 was elected leader, sending a full tablet report...
W20260810 21:00:07.529563 23827 tablet.cc:2369] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:07.578241 24017 raft_consensus.cc:482] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:07.578447 24017 raft_consensus.cc:504] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.578948 24017 leader_election.cc:288] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:07.581863 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.579144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:36313 (local address 127.14.200.1:41493)
0810 21:00:07.579227 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.579229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.579239 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:07.579271 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.579272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.579283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.579335 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.579551 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.580088 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581119 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.581480 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581570 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.581580 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.581759 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.581794 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 21:00:07.581893 23862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.579094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41493 (local address 127.14.200.3:36313)
0810 21:00:07.579162 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.579173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.579218 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.579363 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.579363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.579365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.579437 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.579440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.580174 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.580175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.581036 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.581039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.581568 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.581671 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.581674 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.581748 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.581792 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":25}
I20260810 21:00:07.582039 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.579372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:41007 (local address 127.14.200.2:45565)
0810 21:00:07.579421 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.579424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.579447 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:00:07.579471 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.579473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.579482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.579533 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.579699 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.580169 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581148 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.581664 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581745 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.581755 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.581922 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.581978 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:00:07.582037 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.579158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.14.200.3:41007)
0810 21:00:07.579339 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.579351 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.579418 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.579555 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.579556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.579558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.579624 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.579626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.580245 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.580245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.581045 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.581048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.581760 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.581761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.581841 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.581844 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.581917 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.581956 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260810 21:00:07.582296 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4e3c70611a4888ab675577f496f642" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c" is_pre_election: true
I20260810 21:00:07.582388 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4e3c70611a4888ab675577f496f642" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0" is_pre_election: true
I20260810 21:00:07.582472 23630 raft_consensus.cc:2454] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:07.582576 23778 raft_consensus.cc:2454] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:07.582718 23853 leader_election.cc:302] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:07.582901 24017 raft_consensus.cc:2781] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:07.582942 24017 raft_consensus.cc:482] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:07.582957 24017 raft_consensus.cc:3037] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.583874 24017 raft_consensus.cc:504] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.584017 24017 leader_election.cc:288] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:07.584161 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4e3c70611a4888ab675577f496f642" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c"
I20260810 21:00:07.584200 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4e3c70611a4888ab675577f496f642" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0"
I20260810 21:00:07.584249 23630 raft_consensus.cc:3037] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.584276 23778 raft_consensus.cc:3037] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.585049 23778 raft_consensus.cc:2454] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:07.585079 23630 raft_consensus.cc:2454] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:07.585263 23853 leader_election.cc:302] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:07.585378 24017 raft_consensus.cc:2781] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:07.585530 24017 raft_consensus.cc:686] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Becoming Leader. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Running, Role: LEADER
I20260810 21:00:07.585603 24017 consensus_queue.cc:237] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } }
I20260810 21:00:07.586460 23457 catalog_manager.cc:5495] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } health_report { overall_health: UNKNOWN } } }
I20260810 21:00:07.606282 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.603686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.14.200.3:37747)
0810 21:00:07.603747 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.603749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.603762 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:00:07.603868 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.603869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.603895 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.603973 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:07.604123 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.604628 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.605485 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.605681 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.605758 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.605949 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.606056 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.606151 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.606175 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.606196 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 21:00:07.608175 23630 raft_consensus.cc:1261] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.608187 23778 raft_consensus.cc:1261] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.608565 24017 consensus_queue.cc:1035] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.608729 24019 consensus_queue.cc:1035] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.611908 24024 mvcc.cc:205] Tried to move back new op lower bound from 7317076408760516608 to 7317076408671358976. Current Snapshot: MvccSnapshot[applied={T|T < 7317076408760516608}]
I20260810 21:00:07.613492 23457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:07.613585 23457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:07.614369 23457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Sending DeleteTablet for 3 replicas of tablet 8a4e3c70611a4888ab675577f496f642
I20260810 21:00:07.614704 23608 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4e3c70611a4888ab675577f496f642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:41722
I20260810 21:00:07.614742 23756 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4e3c70611a4888ab675577f496f642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49098
I20260810 21:00:07.615175 23906 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4e3c70611a4888ab675577f496f642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38096
I20260810 21:00:07.615224 24033 tablet_replica.cc:330] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: stopping tablet replica
I20260810 21:00:07.615185 23457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 21:00:07.615320 24033 raft_consensus.cc:2229] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 21:00:07.615320 23457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:07.615391 24032 tablet_replica.cc:330] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: stopping tablet replica
I20260810 21:00:07.615424 24033 raft_consensus.cc:2258] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.615478 24032 raft_consensus.cc:2229] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:07.615680 24032 raft_consensus.cc:2258] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.615855 24033 ts_tablet_manager.cc:1905] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.616032 24034 tablet_replica.cc:330] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: stopping tablet replica
I20260810 21:00:07.616554 24032 ts_tablet_manager.cc:1905] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.617221 24034 raft_consensus.cc:2229] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:07.617288 23756 tablet_service.cc:1467] Processing CreateTablet for tablet 05422b3ea5b6443089f139d3ad95788f (DEFAULT_TABLE table=test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.617406 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05422b3ea5b6443089f139d3ad95788f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.617527 24034 raft_consensus.cc:2258] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.617599 23906 tablet_service.cc:1467] Processing CreateTablet for tablet 05422b3ea5b6443089f139d3ad95788f (DEFAULT_TABLE table=test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.617779 23906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05422b3ea5b6443089f139d3ad95788f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.618737 24010 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Bootstrap starting.
I20260810 21:00:07.618782 23608 tablet_service.cc:1467] Processing CreateTablet for tablet 05422b3ea5b6443089f139d3ad95788f (DEFAULT_TABLE table=test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.618914 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05422b3ea5b6443089f139d3ad95788f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.619184 24010 tablet_bootstrap.cc:654] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.620230 24007 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Bootstrap starting.
I20260810 21:00:07.620268 24008 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Bootstrap starting.
I20260810 21:00:07.620654 24007 tablet_bootstrap.cc:654] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.620688 24008 tablet_bootstrap.cc:654] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.621119 24032 ts_tablet_manager.cc:1918] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:07.621188 24032 log.cc:1196] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/8a4e3c70611a4888ab675577f496f642
I20260810 21:00:07.621371 24033 ts_tablet_manager.cc:1918] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:07.621438 24033 log.cc:1196] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/8a4e3c70611a4888ab675577f496f642
I20260810 21:00:07.621510 24032 ts_tablet_manager.cc:1939] T 8a4e3c70611a4888ab675577f496f642 P 6e8d95c3d8604aecac833a354741032c: Deleting consensus metadata
I20260810 21:00:07.621726 24033 ts_tablet_manager.cc:1939] T 8a4e3c70611a4888ab675577f496f642 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting consensus metadata
I20260810 21:00:07.621971 23441 catalog_manager.cc:4841] TS 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493): tablet 8a4e3c70611a4888ab675577f496f642 (table test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]) successfully deleted
I20260810 21:00:07.622115 23442 catalog_manager.cc:4841] TS 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565): tablet 8a4e3c70611a4888ab675577f496f642 (table test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]) successfully deleted
I20260810 21:00:07.622215 24007 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: No bootstrap required, opened a new log
I20260810 21:00:07.622267 24007 ts_tablet_manager.cc:1397] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:07.622426 24007 raft_consensus.cc:348] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.622509 24007 raft_consensus.cc:374] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.622530 24007 raft_consensus.cc:729] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.622566 24008 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: No bootstrap required, opened a new log
I20260810 21:00:07.622570 24007 consensus_queue.cc:260] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.622604 24008 ts_tablet_manager.cc:1397] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:07.622658 24007 ts_tablet_manager.cc:1428] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:07.622679 24010 tablet_bootstrap.cc:492] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: No bootstrap required, opened a new log
I20260810 21:00:07.622723 24010 ts_tablet_manager.cc:1397] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 21:00:07.622766 24008 raft_consensus.cc:348] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.622835 24008 raft_consensus.cc:374] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.622857 24008 raft_consensus.cc:729] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.622855 24010 raft_consensus.cc:348] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.622901 24008 consensus_queue.cc:260] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.622918 24010 raft_consensus.cc:374] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.622953 24010 raft_consensus.cc:729] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.622980 24008 ts_tablet_manager.cc:1428] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.622996 24010 consensus_queue.cc:260] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.623075 24010 ts_tablet_manager.cc:1428] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.626781 24034 ts_tablet_manager.cc:1905] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.628263 24034 ts_tablet_manager.cc:1918] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:07.628340 24034 log.cc:1196] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/8a4e3c70611a4888ab675577f496f642
I20260810 21:00:07.628608 24034 ts_tablet_manager.cc:1939] T 8a4e3c70611a4888ab675577f496f642 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting consensus metadata
I20260810 21:00:07.629009 23442 catalog_manager.cc:4841] TS ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747): tablet 8a4e3c70611a4888ab675577f496f642 (table test-table [id=268bdd8ce43c40a7bf0e7346bf804cbc]) successfully deleted
W20260810 21:00:07.689931 23979 tablet.cc:2369] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:07.700177 24019 raft_consensus.cc:482] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:07.700291 24019 raft_consensus.cc:504] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.700528 24019 leader_election.cc:288] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:07.700757 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05422b3ea5b6443089f139d3ad95788f" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0" is_pre_election: true
I20260810 21:00:07.700759 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05422b3ea5b6443089f139d3ad95788f" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c" is_pre_election: true
I20260810 21:00:07.700896 23630 raft_consensus.cc:2454] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:07.700902 23778 raft_consensus.cc:2454] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:07.701066 23853 leader_election.cc:302] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:07.701156 24019 raft_consensus.cc:2781] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:07.701195 24019 raft_consensus.cc:482] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:07.701224 24019 raft_consensus.cc:3037] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.701980 24019 raft_consensus.cc:504] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.702140 24019 leader_election.cc:288] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:07.702302 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05422b3ea5b6443089f139d3ad95788f" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c"
I20260810 21:00:07.702428 23630 raft_consensus.cc:3037] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.702365 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05422b3ea5b6443089f139d3ad95788f" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0"
I20260810 21:00:07.702468 23778 raft_consensus.cc:3037] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.703138 23630 raft_consensus.cc:2454] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:07.703307 23853 leader_election.cc:302] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:07.703327 23778 raft_consensus.cc:2454] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:07.703399 24019 raft_consensus.cc:2781] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:07.703449 24019 raft_consensus.cc:686] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Becoming Leader. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Running, Role: LEADER
I20260810 21:00:07.703503 24019 consensus_queue.cc:237] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.704125 23457 catalog_manager.cc:5495] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } health_report { overall_health: HEALTHY } } }
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395606, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.62@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.744387 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.737727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.14.200.62:41297)
0810 21:00:07.737790 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.737793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.737937 (+ 144us) server_negotiation.cc:407] Connection header received
0810 21:00:07.738016 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.738017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.738031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.738099 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.738609 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.739094 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.739756 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.739931 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.739980 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.742041 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.742042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.742043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.742049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.743463 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.743789 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.743790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.743790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.743840 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.744001 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.744002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.744002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.744098 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.744145 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.744261 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.744322 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 21:00:07.744524 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.737638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41297 (local address 127.0.0.1:51256)
0810 21:00:07.737849 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.737877 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.737950 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.738114 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.738114 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.738478 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:07.738531 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.738534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.739163 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.739164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.739686 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.739687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.739994 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.739995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.740037 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.740509 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:07.740513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:07.741921 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:07.743535 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.743543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.743545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.743726 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.743895 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:07.743895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:07.743896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:07.743950 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:07.744149 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:07.744155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:07.744198 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.744247 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.744371 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":42,"thread_start_us":90,"threads_started":1}
I20260810 21:00:07.746460 23457 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51256:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 21:00:07.746692 23457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:07.748174 23756 tablet_service.cc:1467] Processing CreateTablet for tablet 418e74a407754a7eb36e993001fbd19b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b999d1b97ec43de9fcaee698c4e55c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 21:00:07.748198 23608 tablet_service.cc:1467] Processing CreateTablet for tablet 418e74a407754a7eb36e993001fbd19b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b999d1b97ec43de9fcaee698c4e55c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 21:00:07.748216 23906 tablet_service.cc:1467] Processing CreateTablet for tablet 418e74a407754a7eb36e993001fbd19b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b999d1b97ec43de9fcaee698c4e55c6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 21:00:07.748358 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418e74a407754a7eb36e993001fbd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.748358 23906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418e74a407754a7eb36e993001fbd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.748358 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418e74a407754a7eb36e993001fbd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.749681 24010 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: Bootstrap starting.
I20260810 21:00:07.749897 24007 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: Bootstrap starting.
I20260810 21:00:07.750154 24010 tablet_bootstrap.cc:654] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.750308 24007 tablet_bootstrap.cc:654] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.750680 24008 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: Bootstrap starting.
I20260810 21:00:07.750921 24010 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: No bootstrap required, opened a new log
I20260810 21:00:07.750982 24010 ts_tablet_manager.cc:1397] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:07.751024 24007 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: No bootstrap required, opened a new log
I20260810 21:00:07.751046 24008 tablet_bootstrap.cc:654] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.751078 24007 ts_tablet_manager.cc:1397] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:07.751186 24010 raft_consensus.cc:348] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.751225 24007 raft_consensus.cc:348] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.751288 24010 raft_consensus.cc:374] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.751298 24007 raft_consensus.cc:374] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.751315 24010 raft_consensus.cc:729] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.751320 24007 raft_consensus.cc:729] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.751369 24007 consensus_queue.cc:260] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.751369 24010 consensus_queue.cc:260] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.751461 24010 ts_tablet_manager.cc:1428] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:07.751459 24007 ts_tablet_manager.cc:1428] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:07.751639 24008 tablet_bootstrap.cc:492] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: No bootstrap required, opened a new log
I20260810 21:00:07.751624 24015 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.751626 24016 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.751682 24008 ts_tablet_manager.cc:1397] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:07.751700 24015 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.751705 24016 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.751792 24008 raft_consensus.cc:348] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.751863 24008 raft_consensus.cc:374] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.751884 24008 raft_consensus.cc:729] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.751925 24008 consensus_queue.cc:260] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.752014 24008 ts_tablet_manager.cc:1428] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.752045 24019 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.752152 24019 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.798955 24016 raft_consensus.cc:482] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:07.799050 24016 raft_consensus.cc:504] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.799420 24016 leader_election.cc:288] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747)
I20260810 21:00:07.803289 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.799700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:56131 (local address 127.14.200.3:37747)
0810 21:00:07.799746 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.799748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.799809 (+ 61us) server_negotiation.cc:407] Connection header received
0810 21:00:07.799869 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.799869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.799881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.799948 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.800160 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.800919 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.802384 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.802919 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.802998 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.803006 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.803177 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.803217 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:00:07.803294 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.799673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:38305 (local address 127.14.200.1:41493)
0810 21:00:07.799729 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.799731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.799815 (+ 84us) server_negotiation.cc:407] Connection header received
0810 21:00:07.799868 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.799869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.799880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.799951 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:07.800159 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.800918 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.802384 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.802919 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.802998 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.803007 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.803177 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.803219 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:00:07.803329 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.799595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:37747 (local address 127.14.200.2:56131)
0810 21:00:07.799728 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.799740 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.799810 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.799967 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.799968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.799970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.800084 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.800086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.801003 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.801004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.802294 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.802297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.803019 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.803020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.803092 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.803099 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.803157 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.803214 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":19,"thread_start_us":67,"threads_started":1}
I20260810 21:00:07.803332 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.799595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41493 (local address 127.14.200.2:38305)
0810 21:00:07.799735 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.799744 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.799810 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.799971 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.799972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.799974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:07.800085 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.800086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.801003 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.801004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.802294 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.802297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.803014 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.803014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.803092 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.803100 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.803157 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.803214 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":121,"thread_start_us":81,"threads_started":1}
I20260810 21:00:07.803610 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418e74a407754a7eb36e993001fbd19b" candidate_uuid: "847cb1cc324f421e966123f6eb8a0ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" is_pre_election: true
I20260810 21:00:07.803689 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418e74a407754a7eb36e993001fbd19b" candidate_uuid: "847cb1cc324f421e966123f6eb8a0ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c" is_pre_election: true
I20260810 21:00:07.803785 23630 raft_consensus.cc:2454] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847cb1cc324f421e966123f6eb8a0ad0 in term 0.
I20260810 21:00:07.803784 23926 raft_consensus.cc:2454] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 847cb1cc324f421e966123f6eb8a0ad0 in term 0.
I20260810 21:00:07.803970 23704 leader_election.cc:302] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, 847cb1cc324f421e966123f6eb8a0ad0, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:07.804131 24016 raft_consensus.cc:2781] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:07.804175 24016 raft_consensus.cc:482] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:07.804200 24016 raft_consensus.cc:3037] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.805009 24016 raft_consensus.cc:504] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.805164 24016 leader_election.cc:288] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747)
I20260810 21:00:07.805333 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418e74a407754a7eb36e993001fbd19b" candidate_uuid: "847cb1cc324f421e966123f6eb8a0ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c"
I20260810 21:00:07.805418 23630 raft_consensus.cc:3037] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.806042 23630 raft_consensus.cc:2454] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847cb1cc324f421e966123f6eb8a0ad0 in term 1.
I20260810 21:00:07.806175 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418e74a407754a7eb36e993001fbd19b" candidate_uuid: "847cb1cc324f421e966123f6eb8a0ad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208"
I20260810 21:00:07.806257 23926 raft_consensus.cc:3037] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:07.806242 23703 leader_election.cc:302] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, 847cb1cc324f421e966123f6eb8a0ad0; no voters:
I20260810 21:00:07.806342 24016 raft_consensus.cc:2781] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:07.806486 24016 raft_consensus.cc:686] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 LEADER]: Becoming Leader. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Running, Role: LEADER
I20260810 21:00:07.806581 24016 consensus_queue.cc:237] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.807096 24051 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: TxnStatusTablet state changed. Reason: New leader 847cb1cc324f421e966123f6eb8a0ad0. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.807192 23926 raft_consensus.cc:2454] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 847cb1cc324f421e966123f6eb8a0ad0 in term 1.
I20260810 21:00:07.807206 24051 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: This TxnStatusTablet replica's current role is: LEADER
I20260810 21:00:07.807287 23458 catalog_manager.cc:5495] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } health_report { overall_health: UNKNOWN } } }
I20260810 21:00:07.807418 24053 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 21:00:07.807524 24053 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 21:00:07.826812 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.824035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.14.200.2:45565)
0810 21:00:07.824088 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.824090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.824101 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:00:07.824180 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.824181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.824202 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.824274 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:07.824374 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.825183 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.825904 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.826221 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.826293 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.826456 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.826586 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.826661 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.826711 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.826731 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 21:00:07.826833 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.823955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.0.0.1:49100)
0810 21:00:07.824036 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.824050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.824111 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.824284 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.824284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.824286 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 21:00:07.824331 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.824333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.825257 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.825258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.825818 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.825820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.826298 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.826299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.826346 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.826362 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.826650 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.826657 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.826701 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.826740 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 21:00:07.827978 23630 raft_consensus.cc:1261] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Refusing update from remote peer 847cb1cc324f421e966123f6eb8a0ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.828097 23926 raft_consensus.cc:1261] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Refusing update from remote peer 847cb1cc324f421e966123f6eb8a0ad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.828408 24016 consensus_queue.cc:1035] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.828552 24051 consensus_queue.cc:1035] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.829173 24015 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: TxnStatusTablet state changed. Reason: New leader 847cb1cc324f421e966123f6eb8a0ad0. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.829272 24015 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.829247 24019 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: TxnStatusTablet state changed. Reason: New leader 847cb1cc324f421e966123f6eb8a0ad0. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.829334 24019 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.830029 24016 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.830113 24016 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: This TxnStatusTablet replica's current role is: LEADER
I20260810 21:00:07.830524 24051 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.830614 24051 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 847cb1cc324f421e966123f6eb8a0ad0: This TxnStatusTablet replica's current role is: LEADER
I20260810 21:00:07.830801 24015 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.830885 24015 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P 6e8d95c3d8604aecac833a354741032c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.831117 24022 mvcc.cc:205] Tried to move back new op lower bound from 7317076409661317120 to 7317076409576312832. Current Snapshot: MvccSnapshot[applied={T|T < 7317076409661317120}]
I20260810 21:00:07.831218 24019 tablet_replica.cc:439] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "847cb1cc324f421e966123f6eb8a0ad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } } }
I20260810 21:00:07.831275 24019 tablet_replica.cc:442] T 418e74a407754a7eb36e993001fbd19b P ba8aee1ea0ba44fd9e24d67424fcf208: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 21:00:07.836975 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.834446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.14.200.3:37477)
0810 21:00:07.834523 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.834536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.834587 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.834702 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.834703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.834704 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 21:00:07.834743 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.834744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.835592 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.835593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.836139 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.836141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.836603 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.836604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.836644 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.836658 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.836847 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.836849 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.836883 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.836922 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 21:00:07.836983 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.834519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:37477 (local address 127.14.200.2:45565)
0810 21:00:07.834561 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.834563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.834572 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:07.834630 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.834631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.834640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.834701 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:07.834778 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.835514 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.836216 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.836518 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.836587 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.836733 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.836768 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.836842 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.836896 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.836915 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:00:07.839457 23630 raft_consensus.cc:1261] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.839457 23778 raft_consensus.cc:1261] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:07.839674 24017 consensus_queue.cc:1035] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.839756 24019 consensus_queue.cc:1035] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:07.847910 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.845320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:58811 (local address 127.14.200.3:37747)
0810 21:00:07.845366 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.845368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.845541 (+ 173us) server_negotiation.cc:407] Connection header received
0810 21:00:07.845598 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.845599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.845609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.845680 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:07.845848 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.846366 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.847137 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.847341 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.847423 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.847670 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.847704 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.847756 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.847810 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.847834 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:00:07.848026 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.845250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:37747 (local address 127.14.200.2:58811)
0810 21:00:07.845447 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:07.845458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:07.845537 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:07.845682 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:07.845683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:07.845684 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 21:00:07.845753 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.845754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.846460 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.846461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.847054 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:07.847056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.847453 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.847454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:07.847531 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.847545 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 21:00:07.847756 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 21:00:07.847758 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:07.847847 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 21:00:07.847891 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
I20260810 21:00:07.857333 23456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:07.857461 23456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:07.858212 23456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Sending DeleteTablet for 3 replicas of tablet 05422b3ea5b6443089f139d3ad95788f
I20260810 21:00:07.858497 23906 tablet_service.cc:1514] Processing DeleteTablet for tablet 05422b3ea5b6443089f139d3ad95788f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38096
I20260810 21:00:07.858594 24034 tablet_replica.cc:330] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: stopping tablet replica
I20260810 21:00:07.858568 23756 tablet_service.cc:1514] Processing DeleteTablet for tablet 05422b3ea5b6443089f139d3ad95788f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49098
I20260810 21:00:07.858584 23608 tablet_service.cc:1514] Processing DeleteTablet for tablet 05422b3ea5b6443089f139d3ad95788f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:07 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:41722
I20260810 21:00:07.858644 24034 raft_consensus.cc:2229] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:07.858660 24033 tablet_replica.cc:330] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: stopping tablet replica
I20260810 21:00:07.858681 24032 tablet_replica.cc:330] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: stopping tablet replica
I20260810 21:00:07.858703 24034 raft_consensus.cc:2258] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.858704 24033 raft_consensus.cc:2229] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:07.858741 24033 raft_consensus.cc:2258] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.858741 24032 raft_consensus.cc:2229] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:07.858777 24032 raft_consensus.cc:2258] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:07.859047 24034 ts_tablet_manager.cc:1905] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.859076 24033 ts_tablet_manager.cc:1905] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.859118 24032 ts_tablet_manager.cc:1905] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:07.862612 24032 ts_tablet_manager.cc:1918] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 21:00:07.862684 24032 log.cc:1196] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/05422b3ea5b6443089f139d3ad95788f
I20260810 21:00:07.862910 24032 ts_tablet_manager.cc:1939] T 05422b3ea5b6443089f139d3ad95788f P 6e8d95c3d8604aecac833a354741032c: Deleting consensus metadata
I20260810 21:00:07.863225 23441 catalog_manager.cc:4841] TS 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493): tablet 05422b3ea5b6443089f139d3ad95788f (table test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]) successfully deleted
I20260810 21:00:07.863379 24033 ts_tablet_manager.cc:1918] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 21:00:07.863456 24033 log.cc:1196] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/05422b3ea5b6443089f139d3ad95788f
I20260810 21:00:07.863675 24033 ts_tablet_manager.cc:1939] T 05422b3ea5b6443089f139d3ad95788f P 847cb1cc324f421e966123f6eb8a0ad0: Deleting consensus metadata
I20260810 21:00:07.863971 23442 catalog_manager.cc:4841] TS 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565): tablet 05422b3ea5b6443089f139d3ad95788f (table test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]) successfully deleted
I20260810 21:00:07.864144 24034 ts_tablet_manager.cc:1918] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 21:00:07.864208 24034 log.cc:1196] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/05422b3ea5b6443089f139d3ad95788f
I20260810 21:00:07.864416 24034 ts_tablet_manager.cc:1939] T 05422b3ea5b6443089f139d3ad95788f P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting consensus metadata
I20260810 21:00:07.864725 23442 catalog_manager.cc:4841] TS ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747): tablet 05422b3ea5b6443089f139d3ad95788f (table test-table [id=99f0c74edde84fbc80b1f1a9aa00c240]) successfully deleted
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 21:00:07.887916 23514 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:07.881993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.14.200.62:41297)
0810 21:00:07.882057 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.882060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.882191 (+ 131us) server_negotiation.cc:407] Connection header received
0810 21:00:07.882233 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.882234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.882265 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.882334 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.883034 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.883753 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.884467 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.884735 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.884828 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.887849 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51268: BlockingRecv error: recv got EOF from 127.0.0.1:51268 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 21:00:07 dist-test-slave-1x4g krb5kdc[23409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM
I20260810 21:00:07.896157 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:07.888880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.14.200.62:41297)
0810 21:00:07.888916 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:07.888918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:07.889072 (+ 154us) server_negotiation.cc:407] Connection header received
0810 21:00:07.889097 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:07.889098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:07.889109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:07.889162 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:07.889579 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.890282 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.890906 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:07.891203 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:07.891270 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:07.893482 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:07.893483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:07.893483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:07.893490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:07.894884 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.895344 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.895345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.895345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.895399 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:07.895608 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:07.895608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:07.895609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:07.895699 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:07.895766 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:07.895960 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 21:00:07.896058 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:00:07.897953 23456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:00:07.898145 23456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:07.899631 23906 tablet_service.cc:1467] Processing CreateTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 (DEFAULT_TABLE table=test-table [id=5cc19ca34c8240e5b18fda730d26a315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.899662 23608 tablet_service.cc:1467] Processing CreateTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 (DEFAULT_TABLE table=test-table [id=5cc19ca34c8240e5b18fda730d26a315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.899662 23756 tablet_service.cc:1467] Processing CreateTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 (DEFAULT_TABLE table=test-table [id=5cc19ca34c8240e5b18fda730d26a315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:07.899780 23906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a3dcb207254653a578bd2a59ed8b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.899787 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a3dcb207254653a578bd2a59ed8b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.899789 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70a3dcb207254653a578bd2a59ed8b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:07.901166 24010 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Bootstrap starting.
I20260810 21:00:07.901238 24007 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Bootstrap starting.
I20260810 21:00:07.901688 24007 tablet_bootstrap.cc:654] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.901688 24010 tablet_bootstrap.cc:654] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.902069 24008 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Bootstrap starting.
I20260810 21:00:07.902410 24008 tablet_bootstrap.cc:654] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:07.902663 24010 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: No bootstrap required, opened a new log
I20260810 21:00:07.902710 24010 ts_tablet_manager.cc:1397] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:07.902879 24007 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: No bootstrap required, opened a new log
I20260810 21:00:07.902874 24010 raft_consensus.cc:348] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.902928 24007 ts_tablet_manager.cc:1397] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:07.902956 24010 raft_consensus.cc:374] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.902979 24010 raft_consensus.cc:729] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.903034 24010 consensus_queue.cc:260] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.903082 24007 raft_consensus.cc:348] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.903136 24010 ts_tablet_manager.cc:1428] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.903152 24007 raft_consensus.cc:374] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.903172 24007 raft_consensus.cc:729] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.903215 24007 consensus_queue.cc:260] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.903312 24007 ts_tablet_manager.cc:1428] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:07.903944 24008 tablet_bootstrap.cc:492] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: No bootstrap required, opened a new log
I20260810 21:00:07.904006 24008 ts_tablet_manager.cc:1397] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:07.904145 24008 raft_consensus.cc:348] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.904222 24008 raft_consensus.cc:374] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:07.904243 24008 raft_consensus.cc:729] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Initialized, Role: FOLLOWER
I20260810 21:00:07.904284 24008 consensus_queue.cc:260] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:07.904386 24008 ts_tablet_manager.cc:1428] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:08.050892 24019 raft_consensus.cc:482] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:08.050999 24019 raft_consensus.cc:504] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.051250 24019 leader_election.cc:288] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:08.051520 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a3dcb207254653a578bd2a59ed8b30" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0" is_pre_election: true
I20260810 21:00:08.051543 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a3dcb207254653a578bd2a59ed8b30" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c" is_pre_election: true
I20260810 21:00:08.051625 23778 raft_consensus.cc:2454] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:08.051630 23630 raft_consensus.cc:2454] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 0.
I20260810 21:00:08.051806 23854 leader_election.cc:302] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 847cb1cc324f421e966123f6eb8a0ad0, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:08.051911 24019 raft_consensus.cc:2781] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:08.051954 24019 raft_consensus.cc:482] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:08.051973 24019 raft_consensus.cc:3037] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.052691 24019 raft_consensus.cc:504] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.052853 24019 leader_election.cc:288] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493), 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565)
I20260810 21:00:08.053012 23630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a3dcb207254653a578bd2a59ed8b30" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8d95c3d8604aecac833a354741032c"
I20260810 21:00:08.053038 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a3dcb207254653a578bd2a59ed8b30" candidate_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0"
I20260810 21:00:08.053130 23778 raft_consensus.cc:3037] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.053143 23630 raft_consensus.cc:3037] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.053788 23630 raft_consensus.cc:2454] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:08.053791 23778 raft_consensus.cc:2454] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba8aee1ea0ba44fd9e24d67424fcf208 in term 1.
I20260810 21:00:08.053993 23853 leader_election.cc:302] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:08.054103 24019 raft_consensus.cc:2781] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:08.054170 24019 raft_consensus.cc:686] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Becoming Leader. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Running, Role: LEADER
I20260810 21:00:08.054224 24019 consensus_queue.cc:237] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.054889 23456 catalog_manager.cc:5495] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } health_report { overall_health: HEALTHY } } }
I20260810 21:00:08.088580 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.086179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.14.200.3:37747)
0810 21:00:08.086232 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.086234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.086244 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:08.086296 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.086298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.086312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.086361 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:08.086514 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.086994 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.087787 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.088066 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.088143 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.088323 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.088359 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.088399 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.088470 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.088492 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:00:08.089497 23778 raft_consensus.cc:1261] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:08.089510 23630 raft_consensus.cc:1261] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Refusing update from remote peer ba8aee1ea0ba44fd9e24d67424fcf208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:08.089749 24017 consensus_queue.cc:1035] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:08.089905 24019 consensus_queue.cc:1035] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:08.092268 23456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:08.092344 23456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:08.092923 23456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Sending DeleteTablet for 3 replicas of tablet 70a3dcb207254653a578bd2a59ed8b30
I20260810 21:00:08.093174 23756 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49098
I20260810 21:00:08.093209 23608 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:41722
I20260810 21:00:08.093273 24033 tablet_replica.cc:330] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: stopping tablet replica
I20260810 21:00:08.093295 23906 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a3dcb207254653a578bd2a59ed8b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38096
I20260810 21:00:08.093326 24033 raft_consensus.cc:2229] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:08.093370 24032 tablet_replica.cc:330] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: stopping tablet replica
I20260810 21:00:08.093384 24033 raft_consensus.cc:2258] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.093426 24032 raft_consensus.cc:2229] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:08.093391 23456 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 21:00:08.093488 24032 raft_consensus.cc:2258] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.093556 24034 tablet_replica.cc:330] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: stopping tablet replica
W20260810 21:00:08.093560 23456 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:08.093614 24034 raft_consensus.cc:2229] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:08.093683 24034 raft_consensus.cc:2258] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.094064 24032 ts_tablet_manager.cc:1905] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:08.094066 24033 ts_tablet_manager.cc:1905] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:08.094866 23608 tablet_service.cc:1467] Processing CreateTablet for tablet 5eeb36306e854f5f86cc41650c985979 (DEFAULT_TABLE table=test-table [id=09335a6b42ae47d7b366144e73b43cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:08.094872 23906 tablet_service.cc:1467] Processing CreateTablet for tablet 5eeb36306e854f5f86cc41650c985979 (DEFAULT_TABLE table=test-table [id=09335a6b42ae47d7b366144e73b43cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:08.095008 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eeb36306e854f5f86cc41650c985979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:08.095009 23906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eeb36306e854f5f86cc41650c985979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:08.095649 24033 ts_tablet_manager.cc:1918] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:08.095711 24033 log.cc:1196] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/70a3dcb207254653a578bd2a59ed8b30
I20260810 21:00:08.095739 24032 ts_tablet_manager.cc:1918] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:08.095791 24032 log.cc:1196] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/70a3dcb207254653a578bd2a59ed8b30
I20260810 21:00:08.095798 23756 tablet_service.cc:1467] Processing CreateTablet for tablet 5eeb36306e854f5f86cc41650c985979 (DEFAULT_TABLE table=test-table [id=09335a6b42ae47d7b366144e73b43cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:08.095902 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eeb36306e854f5f86cc41650c985979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:08.095947 24033 ts_tablet_manager.cc:1939] T 70a3dcb207254653a578bd2a59ed8b30 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting consensus metadata
I20260810 21:00:08.095999 24032 ts_tablet_manager.cc:1939] T 70a3dcb207254653a578bd2a59ed8b30 P 6e8d95c3d8604aecac833a354741032c: Deleting consensus metadata
I20260810 21:00:08.096472 23442 catalog_manager.cc:4841] TS 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565): tablet 70a3dcb207254653a578bd2a59ed8b30 (table test-table [id=5cc19ca34c8240e5b18fda730d26a315]) successfully deleted
I20260810 21:00:08.096544 23441 catalog_manager.cc:4841] TS 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493): tablet 70a3dcb207254653a578bd2a59ed8b30 (table test-table [id=5cc19ca34c8240e5b18fda730d26a315]) successfully deleted
I20260810 21:00:08.096760 24008 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Bootstrap starting.
I20260810 21:00:08.097289 24008 tablet_bootstrap.cc:654] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:08.097731 24007 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: Bootstrap starting.
I20260810 21:00:08.098021 24008 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: No bootstrap required, opened a new log
I20260810 21:00:08.098079 24008 ts_tablet_manager.cc:1397] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:08.098204 24008 raft_consensus.cc:348] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.098284 24008 raft_consensus.cc:374] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:08.098275 24007 tablet_bootstrap.cc:654] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:08.098310 24008 raft_consensus.cc:729] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba8aee1ea0ba44fd9e24d67424fcf208, State: Initialized, Role: FOLLOWER
I20260810 21:00:08.098353 24008 consensus_queue.cc:260] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.098469 24008 ts_tablet_manager.cc:1428] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:08.098649 24010 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Bootstrap starting.
I20260810 21:00:08.099004 24010 tablet_bootstrap.cc:654] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:08.099218 24007 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: No bootstrap required, opened a new log
I20260810 21:00:08.099278 24007 ts_tablet_manager.cc:1397] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 21:00:08.099438 24007 raft_consensus.cc:348] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.099530 24007 raft_consensus.cc:374] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:08.099556 24007 raft_consensus.cc:729] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Initialized, Role: FOLLOWER
I20260810 21:00:08.099570 24010 tablet_bootstrap.cc:492] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: No bootstrap required, opened a new log
I20260810 21:00:08.099603 24010 ts_tablet_manager.cc:1397] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:08.099596 24007 consensus_queue.cc:260] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.099694 24007 ts_tablet_manager.cc:1428] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:08.099747 24010 raft_consensus.cc:348] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.099818 24010 raft_consensus.cc:374] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:08.099833 24010 raft_consensus.cc:729] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 847cb1cc324f421e966123f6eb8a0ad0, State: Initialized, Role: FOLLOWER
I20260810 21:00:08.099885 24010 consensus_queue.cc:260] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.099989 24010 ts_tablet_manager.cc:1428] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:08.100453 24034 ts_tablet_manager.cc:1905] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:08.102109 24034 ts_tablet_manager.cc:1918] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:08.102186 24034 log.cc:1196] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/70a3dcb207254653a578bd2a59ed8b30
I20260810 21:00:08.102419 24034 ts_tablet_manager.cc:1939] T 70a3dcb207254653a578bd2a59ed8b30 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting consensus metadata
I20260810 21:00:08.102694 23442 catalog_manager.cc:4841] TS ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747): tablet 70a3dcb207254653a578bd2a59ed8b30 (table test-table [id=5cc19ca34c8240e5b18fda730d26a315]) successfully deleted
W20260810 21:00:08.119923 23679 tablet.cc:2369] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:08.218119 24015 raft_consensus.cc:482] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:08.218232 24015 raft_consensus.cc:504] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.218624 24015 leader_election.cc:288] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565), ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747)
I20260810 21:00:08.221807 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.218889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:35739 (local address 127.14.200.2:45565)
0810 21:00:08.218966 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.218969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.219101 (+ 132us) server_negotiation.cc:407] Connection header received
0810 21:00:08.219182 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.219184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.219196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.219255 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:08.219420 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.219943 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221043 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.221406 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221515 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.221531 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.221689 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.221724 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:00:08.221822 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.218807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.14.200.1:35739)
0810 21:00:08.219008 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:08.219021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:08.219123 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:08.219260 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:08.219260 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:08.219262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:08.219336 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.219337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.220054 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.220055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.220951 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.220954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.221539 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.221615 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.221620 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:00:08.221676 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 21:00:08.221718 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260810 21:00:08.221973 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.219146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53241 (local address 127.14.200.3:37747)
0810 21:00:08.219197 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.219199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.219238 (+ 39us) server_negotiation.cc:407] Connection header received
0810 21:00:08.219343 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.219344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.219353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.219429 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:08.219613 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.220144 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221136 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.221604 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221679 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.221688 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.221844 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.221889 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:00:08.222003 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.219083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:37747 (local address 127.14.200.1:53241)
0810 21:00:08.219186 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:08.219196 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:08.219256 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:08.219436 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:08.219437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:08.219459 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:08.219541 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.219542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.220225 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.220226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.221049 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.221051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.221686 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.221688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.221760 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.221764 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:08.221857 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 21:00:08.221903 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":40,"threads_started":1}
I20260810 21:00:08.222244 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eeb36306e854f5f86cc41650c985979" candidate_uuid: "6e8d95c3d8604aecac833a354741032c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" is_pre_election: true
I20260810 21:00:08.222348 23926 raft_consensus.cc:2454] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8d95c3d8604aecac833a354741032c in term 0.
I20260810 21:00:08.222309 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eeb36306e854f5f86cc41650c985979" candidate_uuid: "6e8d95c3d8604aecac833a354741032c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0" is_pre_election: true
I20260810 21:00:08.222545 23778 raft_consensus.cc:2454] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8d95c3d8604aecac833a354741032c in term 0.
I20260810 21:00:08.222733 23555 leader_election.cc:302] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, ba8aee1ea0ba44fd9e24d67424fcf208; no voters:
I20260810 21:00:08.222868 24015 raft_consensus.cc:2781] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:08.222913 24015 raft_consensus.cc:482] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:08.222936 24015 raft_consensus.cc:3037] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.223632 24015 raft_consensus.cc:504] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.223766 24015 leader_election.cc:288] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [CANDIDATE]: Term 1 election: Requested vote from peers 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565), ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747)
I20260810 21:00:08.223961 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eeb36306e854f5f86cc41650c985979" candidate_uuid: "6e8d95c3d8604aecac833a354741032c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208"
I20260810 21:00:08.223966 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eeb36306e854f5f86cc41650c985979" candidate_uuid: "6e8d95c3d8604aecac833a354741032c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "847cb1cc324f421e966123f6eb8a0ad0"
I20260810 21:00:08.224045 23926 raft_consensus.cc:3037] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.224066 23778 raft_consensus.cc:3037] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:08.224666 23778 raft_consensus.cc:2454] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8d95c3d8604aecac833a354741032c in term 1.
I20260810 21:00:08.224689 23926 raft_consensus.cc:2454] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8d95c3d8604aecac833a354741032c in term 1.
I20260810 21:00:08.224851 23555 leader_election.cc:302] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8d95c3d8604aecac833a354741032c, 847cb1cc324f421e966123f6eb8a0ad0; no voters:
I20260810 21:00:08.224963 24015 raft_consensus.cc:2781] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:08.225160 24015 raft_consensus.cc:686] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 1 LEADER]: Becoming Leader. State: Replica: 6e8d95c3d8604aecac833a354741032c, State: Running, Role: LEADER
I20260810 21:00:08.225250 24015 consensus_queue.cc:237] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } }
I20260810 21:00:08.226030 23456 catalog_manager.cc:5495] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e8d95c3d8604aecac833a354741032c (127.14.200.1). New cstate: current_term: 1 leader_uuid: "6e8d95c3d8604aecac833a354741032c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8d95c3d8604aecac833a354741032c" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 } health_report { overall_health: UNKNOWN } } }
I20260810 21:00:08.250200 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.247759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.14.200.1:41493)
0810 21:00:08.247798 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.247800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.247808 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:08.247854 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.247855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.247880 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.247980 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:08.248125 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.248624 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.249402 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.249604 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.249678 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.249840 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.249964 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.250041 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.250093 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.250114 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:00:08.254793 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.252581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:52999 (local address 127.14.200.2:45565)
0810 21:00:08.252621 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.252623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.252764 (+ 141us) server_negotiation.cc:407] Connection header received
0810 21:00:08.252863 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.252864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.252879 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.252926 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:08.253041 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.253463 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.254145 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.254317 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.254365 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.254558 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.254590 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.254649 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.254713 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.254734 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:00:08.254802 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.252491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:45565 (local address 127.14.200.1:52999)
0810 21:00:08.252699 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:08.252709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:08.252821 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:08.252944 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:08.252945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:08.252946 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 21:00:08.253002 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.253003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.253526 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.253527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.254076 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.254077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.254397 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.254398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.254448 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.254462 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.254673 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.254675 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:08.254709 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 21:00:08.254741 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260810 21:00:08.257205 23778 raft_consensus.cc:1261] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Refusing update from remote peer 6e8d95c3d8604aecac833a354741032c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:08.257232 23926 raft_consensus.cc:1261] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Refusing update from remote peer 6e8d95c3d8604aecac833a354741032c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:08.257512 24015 consensus_queue.cc:1035] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba8aee1ea0ba44fd9e24d67424fcf208" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:08.257652 24094 consensus_queue.cc:1035] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [LEADER]: Connected to new peer: Peer: permanent_uuid: "847cb1cc324f421e966123f6eb8a0ad0" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 45565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:08.264159 24003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.262070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:45455 (local address 127.14.200.1:41493)
0810 21:00:08.262108 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:08.262110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:08.262120 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:08.262189 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:08.262190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:08.262199 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:08.262243 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:08.262336 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.262848 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.263515 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.263692 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.263745 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.263878 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.263913 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.263965 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:08.264037 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 21:00:08.264062 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 21:00:08.264164 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.262034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41493 (local address 127.14.200.2:45455)
0810 21:00:08.262078 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:08.262091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:08.262132 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:08.262244 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:08.262245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:08.262246 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 21:00:08.262288 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.262290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.262913 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.262914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.263437 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:08.263439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:08.263758 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:08.263759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:08.263798 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:08.263804 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 21:00:08.263985 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 21:00:08.263987 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:08.264036 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 21:00:08.264087 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 21:00:08.271310 23458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:08.271420 23458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:08.272132 23458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb79b5305af44341b9fc370292ff5c76: Sending DeleteTablet for 3 replicas of tablet 5eeb36306e854f5f86cc41650c985979
I20260810 21:00:08.272420 23608 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eeb36306e854f5f86cc41650c985979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:41722
I20260810 21:00:08.272420 23906 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eeb36306e854f5f86cc41650c985979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38096
I20260810 21:00:08.272509 24032 tablet_replica.cc:330] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c: stopping tablet replica
I20260810 21:00:08.272531 24034 tablet_replica.cc:330] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: stopping tablet replica
I20260810 21:00:08.272576 24032 raft_consensus.cc:2229] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:08.272583 24034 raft_consensus.cc:2229] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:08.272614 24034 raft_consensus.cc:2258] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.272673 24032 raft_consensus.cc:2258] T 5eeb36306e854f5f86cc41650c985979 P 6e8d95c3d8604aecac833a354741032c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.272787 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 23538
I20260810 21:00:08.273396 23756 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eeb36306e854f5f86cc41650c985979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:08 UTC) from {username='oryx', principal='oryx/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:49098
I20260810 21:00:08.273511 24033 tablet_replica.cc:330] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: stopping tablet replica
I20260810 21:00:08.273510 24034 ts_tablet_manager.cc:1905] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:08.273566 24033 raft_consensus.cc:2229] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:08.273599 24033 raft_consensus.cc:2258] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:08.274268 24033 ts_tablet_manager.cc:1905] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:08.275306 24033 ts_tablet_manager.cc:1918] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 21:00:08.275347 24033 log.cc:1196] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/5eeb36306e854f5f86cc41650c985979
I20260810 21:00:08.275424 24034 ts_tablet_manager.cc:1918] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 21:00:08.275492 24034 log.cc:1196] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/5eeb36306e854f5f86cc41650c985979
I20260810 21:00:08.275511 24033 ts_tablet_manager.cc:1939] T 5eeb36306e854f5f86cc41650c985979 P 847cb1cc324f421e966123f6eb8a0ad0: Deleting consensus metadata
I20260810 21:00:08.275709 24034 ts_tablet_manager.cc:1939] T 5eeb36306e854f5f86cc41650c985979 P ba8aee1ea0ba44fd9e24d67424fcf208: Deleting consensus metadata
I20260810 21:00:08.275921 23442 catalog_manager.cc:4841] TS 847cb1cc324f421e966123f6eb8a0ad0 (127.14.200.2:45565): tablet 5eeb36306e854f5f86cc41650c985979 (table test-table [id=09335a6b42ae47d7b366144e73b43cf1]) successfully deleted
I20260810 21:00:08.276003 23442 catalog_manager.cc:4841] TS ba8aee1ea0ba44fd9e24d67424fcf208 (127.14.200.3:37747): tablet 5eeb36306e854f5f86cc41650c985979 (table test-table [id=09335a6b42ae47d7b366144e73b43cf1]) successfully deleted
W20260810 21:00:08.283205 23441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:41493 (error 108)
I20260810 21:00:08.283492 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 23687
I20260810 21:00:08.289929 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:08.283677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41493 (local address 127.0.0.1:41742)
0810 21:00:08.289838 (+ 6161us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:08.289890 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6139,"thread_start_us":39,"threads_started":1}
W20260810 21:00:08.290122 23441 catalog_manager.cc:4584] TS 6e8d95c3d8604aecac833a354741032c (127.14.200.1:41493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5eeb36306e854f5f86cc41650c985979: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41493: connect: Connection refused (error 111)
I20260810 21:00:08.296060 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 23835
I20260810 21:00:08.303591 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 23422
2026-08-10T21:00:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2723 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:08 dist-test-slave-1x4g krb5kdc[24112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:08 dist-test-slave-1x4g krb5kdc[24112](info): set up 2 sockets
Aug 10 21:00:08 dist-test-slave-1x4g krb5kdc[24112](info): commencing operation
krb5kdc: starting...
W20260810 21:00:09.380635 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:09 dist-test-slave-1x4g krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:00:09.505321 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36795
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:09.588255 24125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:09.588486 24125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:09.588532 24125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:09.589712 24125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:09.589778 24125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:09.589802 24125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:09.589833 24125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:09.589859 24125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:09.591344 24125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:09.591626 24125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:09.591852 24125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:09.596967 24131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:09.596974 24133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:09.597234 24125 server_base.cc:1034] running on GCE node
W20260810 21:00:09.596992 24130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:09.597579 24125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:09.597919 24125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:09.599083 24125 hybrid_clock.cc:648] HybridClock initialized: now 1786395609599062 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T21:00:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1297 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:09 dist-test-slave-1x4g krb5kdc[24139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:09 dist-test-slave-1x4g krb5kdc[24139](info): set up 2 sockets
Aug 10 21:00:09 dist-test-slave-1x4g krb5kdc[24139](info): commencing operation
krb5kdc: starting...
W20260810 21:00:10.711993 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:10 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:00:10.839210 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43519
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:10.922147 24154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:10.922369 24154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:10.922390 24154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:10.923496 24154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:10.923547 24154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:10.923561 24154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:10.923573 24154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:10.923623 24154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:10.925130 24154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:10.925397 24154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:10.925632 24154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:10.930603 24163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:10.930665 24160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:10.930699 24154 server_base.cc:1034] running on GCE node
W20260810 21:00:10.930604 24159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:10.931110 24154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:10.931409 24154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:10.932587 24154 hybrid_clock.cc:648] HybridClock initialized: now 1786395610932578 us; error 49 us; skew 500 ppm
Aug 10 21:00:10 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395610, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:10.935184 24154 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:00:10.935662 24154 webserver.cc:460] Webserver started at http://127.14.200.62:46153/ using document root <none> and password file <none>
I20260810 21:00:10.935910 24154 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:10.935956 24154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:10.936122 24154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:10.937263 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "0830388224fe43729c2e5ee21bef20b1"
format_stamp: "Formatted at 2026-08-10 21:00:10 on dist-test-slave-1x4g"
server_key: "35898c40d56bbd61bf93f18d31048314"
server_key_iv: "0201b01f14afbef79cbabf414b4a6735"
server_key_version: "encryptionkey@0"
I20260810 21:00:10.937634 24154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "0830388224fe43729c2e5ee21bef20b1"
format_stamp: "Formatted at 2026-08-10 21:00:10 on dist-test-slave-1x4g"
server_key: "35898c40d56bbd61bf93f18d31048314"
server_key_iv: "0201b01f14afbef79cbabf414b4a6735"
server_key_version: "encryptionkey@0"
I20260810 21:00:10.938994 24154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:10.939739 24171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:10.939956 24154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:10.940050 24154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "0830388224fe43729c2e5ee21bef20b1"
format_stamp: "Formatted at 2026-08-10 21:00:10 on dist-test-slave-1x4g"
server_key: "35898c40d56bbd61bf93f18d31048314"
server_key_iv: "0201b01f14afbef79cbabf414b4a6735"
server_key_version: "encryptionkey@0"
I20260810 21:00:10.940105 24154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:10.963786 24154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:10.966126 24154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:10.966248 24154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:10.970355 24154 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:43519
I20260810 21:00:10.970389 24228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:43519 every 8 connection(s)
I20260810 21:00:10.970774 24154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:10.971705 24229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:10.974004 24229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Bootstrap starting.
I20260810 21:00:10.974583 24229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:10.974826 24229 log.cc:824] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:10.975591 24229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: No bootstrap required, opened a new log
I20260810 21:00:10.975644 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24154
I20260810 21:00:10.975759 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:10.975970 15136 external_mini_cluster.cc:1428] Setting key 1fa3a66aff41974b95b9dba71b2ea93e
I20260810 21:00:10.977769 24229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:10.977950 24229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:10.977989 24229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0830388224fe43729c2e5ee21bef20b1, State: Initialized, Role: FOLLOWER
I20260810 21:00:10.978158 24229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:10.978259 24229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:10.978293 24229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:10.978349 24229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:10.979388 24229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:10.979518 24229 leader_election.cc:302] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0830388224fe43729c2e5ee21bef20b1; no voters:
I20260810 21:00:10.979727 24229 leader_election.cc:288] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:10.979751 24234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:10.979869 24234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 LEADER]: Becoming Leader. State: Replica: 0830388224fe43729c2e5ee21bef20b1, State: Running, Role: LEADER
I20260810 21:00:10.980010 24229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:10.980028 24234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:10.981823 24235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0830388224fe43729c2e5ee21bef20b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } } }
I20260810 21:00:10.981864 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0830388224fe43729c2e5ee21bef20b1. Latest consensus state: current_term: 1 leader_uuid: "0830388224fe43729c2e5ee21bef20b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } } }
I20260810 21:00:10.981909 24235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:10.981945 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: This master's current role is: LEADER
W20260810 21:00:10.982337 24251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:00:10.982393 24251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:00:10.982456 24253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:10.982656 24253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:10.984068 24253 catalog_manager.cc:1347] Generated new cluster ID: 86fb635430a14f8fbe098e3526dc06c0
I20260810 21:00:10.984118 24253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:00:10 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:10.988831 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:10.976550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.14.200.62:43519)
0810 21:00:10.976756 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:10.976763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:10.976852 (+ 89us) server_negotiation.cc:407] Connection header received
0810 21:00:10.977751 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:10.977759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:10.977813 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:10.977932 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:10.978804 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:10.979997 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:10.981228 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:10.981879 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:10.982008 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:10.984962 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:10.984966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:10.984967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:10.984979 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:10.986888 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:10.987273 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:10.987275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:10.987276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:10.987394 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:10.987566 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:10.987566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:10.987566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:10.987834 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:10.987989 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:10.988465 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 21:00:10.988603 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260810 21:00:11.009752 24253 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:11.010481 24253 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:11.014882 24253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Generated new TSK 0
I20260810 21:00:11.015079 24253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:00:11.043792 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:11.127310 24259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.127501 24259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.127523 24259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.128631 24259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.128669 24259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.128731 24259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:11.130173 24259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.130353 24259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.130538 24259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.135356 24265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.135406 24268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.135370 24266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.135664 24259 server_base.cc:1034] running on GCE node
I20260810 21:00:11.135900 24259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.136152 24259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.137292 24259 hybrid_clock.cc:648] HybridClock initialized: now 1786395611137283 us; error 27 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.139791 24259 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:00:11.140233 24259 webserver.cc:460] Webserver started at http://127.14.200.1:45221/ using document root <none> and password file <none>
I20260810 21:00:11.140460 24259 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.140506 24259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.140687 24259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:11.141830 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "8f71117576cf453f80721e4aeb35b1bf"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "57a0dbfc95b3a0862f33b6321d74882a"
server_key_iv: "6a344b217a0463adee86cf59475d173a"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.142210 24259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "8f71117576cf453f80721e4aeb35b1bf"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "57a0dbfc95b3a0862f33b6321d74882a"
server_key_iv: "6a344b217a0463adee86cf59475d173a"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.143747 24259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:11.144516 24275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.144701 24259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:11.144825 24259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "8f71117576cf453f80721e4aeb35b1bf"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "57a0dbfc95b3a0862f33b6321d74882a"
server_key_iv: "6a344b217a0463adee86cf59475d173a"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.144908 24259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:11.171056 24259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:11.173138 24259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:11.173314 24259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:11.173583 24259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:11.173928 24259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:11.173983 24259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.174038 24259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:11.174084 24259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.179875 24259 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42249
I20260810 21:00:11.180034 24402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42249 every 8 connection(s)
I20260810 21:00:11.180312 24259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:11.189196 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.181134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:49825 (local address 127.14.200.62:43519)
0810 21:00:11.181187 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.181190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.181408 (+ 218us) server_negotiation.cc:407] Connection header received
0810 21:00:11.181969 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.181970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.181984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.182049 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.182964 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.183488 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.184467 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.184667 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.184718 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.186836 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:11.186837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:11.186837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:11.186844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:11.188033 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.188373 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.188373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.188374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.188422 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.188624 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.188625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.188625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.188716 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:11.188764 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:11.189058 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 21:00:11.189134 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:00:11.189386 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.181095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:49825)
0810 21:00:11.181274 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.181316 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.181841 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.182109 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.182113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.182690 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.182881 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.182885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.183573 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.183574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.184391 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.184393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.184739 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.184740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.184858 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.185397 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.185407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.186754 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:11.188102 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.188108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.188110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.188310 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.188478 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.188478 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.188478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.188572 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.188768 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:11.188809 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:11.188950 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 21:00:11.189033 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 21:00:11.189148 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260810 21:00:11.189889 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24259
I20260810 21:00:11.190034 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:11.190037 24403 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43519
I20260810 21:00:11.190135 24403 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:11.190281 15136 external_mini_cluster.cc:1428] Setting key 7d8af1d6bf998aac05199c18375ea200
I20260810 21:00:11.190332 24403 heartbeater.cc:511] Master 127.14.200.62:43519 requested a full tablet report, sending...
I20260810 21:00:11.190896 24189 ts_manager.cc:194] Registered new tserver with Master: 8f71117576cf453f80721e4aeb35b1bf (127.14.200.1:42249)
I20260810 21:00:11.191485 24189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:49825
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:00:11.215406 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:11.298651 24409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.298888 24409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.298928 24409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.300084 24409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.300153 24409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.300195 24409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:11.301653 24409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.301923 24409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.302156 24409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.307461 24417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.307451 24419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.307451 24416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.307569 24409 server_base.cc:1034] running on GCE node
I20260810 21:00:11.307898 24409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.308183 24409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.309343 24409 hybrid_clock.cc:648] HybridClock initialized: now 1786395611309323 us; error 35 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.311821 24409 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:00:11.312258 24409 webserver.cc:460] Webserver started at http://127.14.200.2:38055/ using document root <none> and password file <none>
I20260810 21:00:11.312482 24409 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.312551 24409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.312703 24409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:11.313781 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "f612bd996313467e961fb221b93bb092"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "225801d2135590f58d52aac9330067d8"
server_key_iv: "78e75ee2429235a1cdd4f52f480b9342"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.314148 24409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "f612bd996313467e961fb221b93bb092"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "225801d2135590f58d52aac9330067d8"
server_key_iv: "78e75ee2429235a1cdd4f52f480b9342"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.315547 24409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:11.316267 24426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.316447 24409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:11.316561 24409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "f612bd996313467e961fb221b93bb092"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "225801d2135590f58d52aac9330067d8"
server_key_iv: "78e75ee2429235a1cdd4f52f480b9342"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.316649 24409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:11.335731 24409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:11.337816 24409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:11.337975 24409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:11.338256 24409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:11.338690 24409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:11.338737 24409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.338769 24409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:11.338793 24409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.345257 24409 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:42007
I20260810 21:00:11.345367 24549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:42007 every 8 connection(s)
I20260810 21:00:11.345655 24409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:00:11.351377 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24409
I20260810 21:00:11.351498 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:11.351696 15136 external_mini_cluster.cc:1428] Setting key 08722bf8397fbadfa77880e3192a4df2
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:11.355383 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.346321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:57465 (local address 127.14.200.62:43519)
0810 21:00:11.346367 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.346369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.346624 (+ 255us) server_negotiation.cc:407] Connection header received
0810 21:00:11.347094 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.347095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.347104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.347162 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.348111 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.348662 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.349752 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.349955 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.350022 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.352285 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:11.352287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:11.352287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:11.352294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:11.353929 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.354323 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.354324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.354324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.354390 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.354748 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.354749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.354749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.354860 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:11.354916 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:11.355244 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 21:00:11.355320 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:00:11.355676 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.346300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:57465)
0810 21:00:11.346476 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.346518 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.347011 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.347215 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.347218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.347836 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.348026 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.348031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.348850 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.348851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.349681 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.349683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.350041 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.350041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.350142 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.350689 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.350699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.352194 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:11.354014 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.354021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.354023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.354238 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.354489 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.354490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.354490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.354658 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.354928 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:11.354934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:11.355102 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 21:00:11.355224 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 21:00:11.355405 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260810 21:00:11.356205 24550 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43519
I20260810 21:00:11.356313 24550 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:11.356508 24550 heartbeater.cc:511] Master 127.14.200.62:43519 requested a full tablet report, sending...
I20260810 21:00:11.356917 24189 ts_manager.cc:194] Registered new tserver with Master: f612bd996313467e961fb221b93bb092 (127.14.200.2:42007)
I20260810 21:00:11.357303 24189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:57465
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:00:11.379179 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:11.461150 24556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.461382 24556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.461422 24556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.462585 24556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.462656 24556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.462698 24556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:11.464195 24556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.464460 24556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.464701 24556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.469708 24562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.469875 24563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.469766 24556 server_base.cc:1034] running on GCE node
W20260810 21:00:11.469718 24565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.470153 24556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.470407 24556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.471572 24556 hybrid_clock.cc:648] HybridClock initialized: now 1786395611471568 us; error 32 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.474213 24556 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:00:11.474680 24556 webserver.cc:460] Webserver started at http://127.14.200.3:36783/ using document root <none> and password file <none>
I20260810 21:00:11.474915 24556 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.474984 24556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.475153 24556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:11.476233 24556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "8ca180b1707149d5b02466bc3a09814f"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "78636d7b5ea5532a0bdb0350168b7c99"
server_key_iv: "45575e62d41759e22a6373d9ef849cae"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.476620 24556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "8ca180b1707149d5b02466bc3a09814f"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "78636d7b5ea5532a0bdb0350168b7c99"
server_key_iv: "45575e62d41759e22a6373d9ef849cae"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.477998 24556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:11.478688 24572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.478863 24556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.478961 24556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "8ca180b1707149d5b02466bc3a09814f"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "78636d7b5ea5532a0bdb0350168b7c99"
server_key_iv: "45575e62d41759e22a6373d9ef849cae"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.479045 24556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:11.492707 24556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:11.495064 24556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:11.495222 24556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:11.495456 24556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:11.495765 24556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:11.495821 24556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.495863 24556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:11.495890 24556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.501716 24556 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:36011
I20260810 21:00:11.501765 24692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:36011 every 8 connection(s)
I20260810 21:00:11.502082 24556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:11.504534 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24556
I20260810 21:00:11.504630 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:11.504938 15136 external_mini_cluster.cc:1428] Setting key 52494751748f790021f1297a3ca156b3
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:11.511253 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.502771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:41705 (local address 127.14.200.62:43519)
0810 21:00:11.502830 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.502832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.503018 (+ 186us) server_negotiation.cc:407] Connection header received
0810 21:00:11.503540 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.503541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.503553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.503627 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.504709 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.505354 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.506484 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.506669 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.506724 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.508685 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:11.508686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:11.508686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:11.508692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:11.510129 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.510473 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.510474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.510475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.510539 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:11.510750 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:11.510750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:11.510750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:11.510849 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:11.510899 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:11.511129 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 21:00:11.511191 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:00:11.511402 24695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.502719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:41705)
0810 21:00:11.502887 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.502930 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.503414 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.503700 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.503704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.504283 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.504568 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.504576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.505433 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.505434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.506404 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.506407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.506744 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.506745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.506842 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.507252 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.507260 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.508602 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:11.510195 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.510201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.510203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.510409 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.510597 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:11.510598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:11.510598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:11.510693 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:11.510902 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:11.510907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:11.511020 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 21:00:11.511100 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 21:00:11.511214 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260810 21:00:11.511888 24693 heartbeater.cc:347] Connected to a master server at 127.14.200.62:43519
I20260810 21:00:11.511999 24693 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:11.512179 24693 heartbeater.cc:511] Master 127.14.200.62:43519 requested a full tablet report, sending...
I20260810 21:00:11.512590 24189 ts_manager.cc:194] Registered new tserver with Master: 8ca180b1707149d5b02466bc3a09814f (127.14.200.3:36011)
I20260810 21:00:11.513103 24189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:41705
I20260810 21:00:11.516578 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
Entry for principal oryx/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab.
WARNING: no policy specified for oryx/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
Entry for principal oryx/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab.
WARNING: no policy specified for oryx/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
Entry for principal oryx/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab.
I20260810 21:00:11.589490 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24259
I20260810 21:00:11.601223 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24409
I20260810 21:00:11.608170 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24556
I20260810 21:00:11.614576 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24154
I20260810 21:00:11.621932 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43519
--webserver_interface=127.14.200.62
--webserver_port=46153
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:11.704087 24704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.704285 24704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.704306 24704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.705644 24704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:11.705687 24704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.705703 24704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.705742 24704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:11.705771 24704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:11.707238 24704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=46153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.707506 24704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.707741 24704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.712415 24709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.712463 24710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.712411 24712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.712831 24704 server_base.cc:1034] running on GCE node
I20260810 21:00:11.713071 24704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.713342 24704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.714505 24704 hybrid_clock.cc:648] HybridClock initialized: now 1786395611714484 us; error 32 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.717027 24704 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:00:11.717495 24704 webserver.cc:460] Webserver started at http://127.14.200.62:46153/ using document root <none> and password file <none>
I20260810 21:00:11.717720 24704 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.717774 24704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.719069 24704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:11.719655 24719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.719833 24704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.719921 24704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "0830388224fe43729c2e5ee21bef20b1"
format_stamp: "Formatted at 2026-08-10 21:00:10 on dist-test-slave-1x4g"
server_key: "35898c40d56bbd61bf93f18d31048314"
server_key_iv: "0201b01f14afbef79cbabf414b4a6735"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.720263 24704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:11.730248 24704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:11.732125 24704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:11.732276 24704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:11.736361 24704 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:43519
I20260810 21:00:11.736420 24777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:43519 every 8 connection(s)
I20260810 21:00:11.736886 24704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:11.738363 24778 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:00:11.738981 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Bootstrap starting.
I20260810 21:00:11.746974 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24704
I20260810 21:00:11.747381 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:42249
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:43519
--webserver_port=45221
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:00:11.748682 24778 log.cc:824] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:11.750458 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:00:11.750775 24778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Bootstrap complete.
I20260810 21:00:11.753283 24778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:11.753535 24778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0830388224fe43729c2e5ee21bef20b1, State: Initialized, Role: FOLLOWER
I20260810 21:00:11.753731 24778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:11.753813 24778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:11.753866 24778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:11.753919 24778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:00:11.754922 24778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:11.755051 24778 leader_election.cc:302] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0830388224fe43729c2e5ee21bef20b1; no voters:
I20260810 21:00:11.755249 24778 leader_election.cc:288] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 21:00:11.755327 24782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 21:00:11.755551 24778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:11.755587 24782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [term 2 LEADER]: Becoming Leader. State: Replica: 0830388224fe43729c2e5ee21bef20b1, State: Running, Role: LEADER
I20260810 21:00:11.755702 24782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } }
I20260810 21:00:11.756021 24783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0830388224fe43729c2e5ee21bef20b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } } }
I20260810 21:00:11.756018 24782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0830388224fe43729c2e5ee21bef20b1. Latest consensus state: current_term: 2 leader_uuid: "0830388224fe43729c2e5ee21bef20b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0830388224fe43729c2e5ee21bef20b1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 43519 } } }
I20260810 21:00:11.756270 24783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:11.756294 24782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:11.756649 24793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:11.757257 24793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:11.757411 24793 catalog_manager.cc:1259] Loaded cluster ID: 86fb635430a14f8fbe098e3526dc06c0
I20260810 21:00:11.757696 24793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:00:11.759047 24793 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:11.759263 24793 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0830388224fe43729c2e5ee21bef20b1: Loaded TSK: 0
I20260810 21:00:11.759538 24793 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:00:11.833887 24781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.834095 24781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.834118 24781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.835282 24781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.835321 24781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.835350 24781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:11.836887 24781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.1.keytab
--principal=oryx/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:42249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=45221
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.837129 24781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.837373 24781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.843238 24805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.843312 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.843353 24808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.843683 24781 server_base.cc:1034] running on GCE node
I20260810 21:00:11.843918 24781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.844189 24781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.845350 24781 hybrid_clock.cc:648] HybridClock initialized: now 1786395611845338 us; error 32 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.848080 24781 init.cc:377] Logged in from keytab as oryx/127.14.200.1@KRBTEST.COM (short username oryx)
I20260810 21:00:11.848543 24781 webserver.cc:460] Webserver started at http://127.14.200.1:45221/ using document root <none> and password file <none>
I20260810 21:00:11.848848 24781 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.848963 24781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.850360 24781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:11.851045 24815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.851267 24781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:11.851367 24781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "8f71117576cf453f80721e4aeb35b1bf"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "57a0dbfc95b3a0862f33b6321d74882a"
server_key_iv: "6a344b217a0463adee86cf59475d173a"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.851831 24781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:11.871958 24781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:11.874302 24781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:11.874450 24781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:11.874689 24781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:11.875005 24781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:11.875047 24781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.875077 24781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:11.875092 24781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.882318 24781 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42249
I20260810 21:00:11.882375 24935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42249 every 8 connection(s)
I20260810 21:00:11.882777 24781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:11.883795 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24781
I20260810 21:00:11.884130 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:42007
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:43519
--webserver_port=38055
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:11.894308 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.883625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:33007)
0810 21:00:11.884891 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.884942 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.885723 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.886283 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.886286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.886858 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.887045 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.887050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.888564 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.888566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.889975 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.889979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.890754 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.890756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.890912 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.891519 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.891530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.894167 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1218,"thread_start_us":90,"threads_started":1}
W20260810 21:00:11.894558 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:11.883682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33007 (local address 127.14.200.62:43519)
0810 21:00:11.883968 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.883978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.885073 (+ 1095us) server_negotiation.cc:407] Connection header received
0810 21:00:11.885982 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.885996 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.886042 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.886124 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.887141 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.888455 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.890093 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.890648 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.890761 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.894350 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:33007: BlockingRecv error: recv got EOF from 127.14.200.1:33007 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
W20260810 21:00:11.894743 24936 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:43519 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.200.62:43519: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:11.899292 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.894990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:33843)
0810 21:00:11.895046 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.895056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.895102 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.895256 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.895257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.895408 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.895454 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.895455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.896077 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.896078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.896570 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.896571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.897007 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.897008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.897081 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.897507 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.897516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.899227 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 21:00:11.899379 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:11.895055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33843 (local address 127.14.200.62:43519)
0810 21:00:11.895119 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.895121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.895130 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:11.895154 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.895155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.895168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.895252 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.895542 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.895998 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.896675 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.896933 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.897008 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.899305 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:33843: BlockingRecv error: recv got EOF from 127.14.200.1:33843 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:11.903674 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:11.899573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:59609)
0810 21:00:11.899629 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:11.899638 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:11.899668 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:11.899781 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:11.899782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:11.899912 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:11.899943 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.899944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.900468 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.900469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.901253 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:11.901255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.901561 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.901562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:11.901623 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.901957 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:11.901962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:11.903599 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 21:00:11.903723 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:11.899632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:59609 (local address 127.14.200.62:43519)
0810 21:00:11.899666 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:11.899667 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:11.899681 (+ 14us) server_negotiation.cc:407] Connection header received
0810 21:00:11.899707 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:11.899708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:11.899714 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:11.899763 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:11.900005 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.900400 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.901321 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:11.901486 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:11.901536 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:11.903663 (+ 2127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:59609: BlockingRecv error: recv got EOF from 127.14.200.1:59609 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 21:00:11.903792 24936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:00:11.969421 24940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:11.969667 24940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:11.969707 24940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:11.970889 24940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:11.970958 24940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:11.971004 24940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:11.972402 24940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.2.keytab
--principal=oryx/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:42007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=38055
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:11.972666 24940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:11.972945 24940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:11.977914 24946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.977910 24948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:11.977972 24945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:11.978183 24940 server_base.cc:1034] running on GCE node
I20260810 21:00:11.978439 24940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:11.978699 24940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:11.979862 24940 hybrid_clock.cc:648] HybridClock initialized: now 1786395611979856 us; error 37 us; skew 500 ppm
Aug 10 21:00:11 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395611, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:11.986172 24940 init.cc:377] Logged in from keytab as oryx/127.14.200.2@KRBTEST.COM (short username oryx)
I20260810 21:00:11.986660 24940 webserver.cc:460] Webserver started at http://127.14.200.2:38055/ using document root <none> and password file <none>
I20260810 21:00:11.986893 24940 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:11.986979 24940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:11.988314 24940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:11.988963 24955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:11.989135 24940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:11.989243 24940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "f612bd996313467e961fb221b93bb092"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "225801d2135590f58d52aac9330067d8"
server_key_iv: "78e75ee2429235a1cdd4f52f480b9342"
server_key_version: "encryptionkey@0"
I20260810 21:00:11.989607 24940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:12.004352 24940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:12.006460 24940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:12.006639 24940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:12.006887 24940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:12.007206 24940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:12.007262 24940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:12.007304 24940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:12.007351 24940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:12.013363 24940 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:42007
I20260810 21:00:12.013415 25081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:42007 every 8 connection(s)
I20260810 21:00:12.013761 24940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.020483 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 24940
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.020844 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:36011
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:43519
--webserver_port=36783
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:12.021726 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.014468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:57521 (local address 127.14.200.62:43519)
0810 21:00:12.014545 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.014547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.014771 (+ 224us) server_negotiation.cc:407] Connection header received
0810 21:00:12.015425 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.015427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.015446 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.015512 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.016544 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.017139 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.018144 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.018361 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.018448 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.021670 (+ 3222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:57521: BlockingRecv error: recv got EOF from 127.14.200.2:57521 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 21:00:12.021745 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.014461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:57521)
0810 21:00:12.014643 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.014681 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.015219 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.015591 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.015594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.016227 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.016435 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.016441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.017238 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.017239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.018049 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.018051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.018461 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.018462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.018591 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.019056 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.019065 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.021503 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260810 21:00:12.022140 25082 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:43519 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.200.62:43519: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.026911 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.022394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:53395)
0810 21:00:12.022449 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.022461 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.022499 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.022653 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.022654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.022803 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.022848 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.022850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.023501 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.023502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.024014 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.024016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.024396 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.024397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.024477 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.024986 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.024990 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.026829 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 21:00:12.026985 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.022447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:53395 (local address 127.14.200.62:43519)
0810 21:00:12.022495 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.022496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.022505 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:12.022558 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.022559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.022567 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.022633 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.022922 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.023414 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.024124 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.024323 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.024398 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.026902 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:53395: BlockingRecv error: recv got EOF from 127.14.200.2:53395 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.031756 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.027442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:39115)
0810 21:00:12.027567 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.027578 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.027649 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.027791 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.027792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.027914 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.027941 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.027942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.028548 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.028549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.029091 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.029092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.029439 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.029440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.029504 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.029926 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.029930 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.031678 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260810 21:00:12.031812 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.027509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39115 (local address 127.14.200.62:43519)
0810 21:00:12.027557 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.027559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.027634 (+ 75us) server_negotiation.cc:407] Connection header received
0810 21:00:12.027709 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.027710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.027719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.027787 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.028014 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.028467 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.029172 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.029354 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.029438 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.031746 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:39115: BlockingRecv error: recv got EOF from 127.14.200.2:39115 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 21:00:12.032152 25082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:00:12.106374 25085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:12.106609 25085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:12.106648 25085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:12.107803 25085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:12.107867 25085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:12.107910 25085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:12.109480 25085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/krb5kdc/oryx_127.14.200.3.keytab
--principal=oryx/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:36011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=36783
--tserver_master_addrs=127.14.200.62:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:12.109738 25085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:12.109968 25085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:12.114920 25091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:12.114921 25094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:12.115021 25092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:12.114938 25085 server_base.cc:1034] running on GCE node
I20260810 21:00:12.115317 25085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:12.115566 25085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:12.116703 25085 hybrid_clock.cc:648] HybridClock initialized: now 1786395612116686 us; error 35 us; skew 500 ppm
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395612, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:12.119683 25085 init.cc:377] Logged in from keytab as oryx/127.14.200.3@KRBTEST.COM (short username oryx)
I20260810 21:00:12.120069 25085 webserver.cc:460] Webserver started at http://127.14.200.3:36783/ using document root <none> and password file <none>
I20260810 21:00:12.120263 25085 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:12.120316 25085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:12.121694 25085 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:12.122231 25105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:12.122449 25085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:12.122551 25085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "8ca180b1707149d5b02466bc3a09814f"
format_stamp: "Formatted at 2026-08-10 21:00:11 on dist-test-slave-1x4g"
server_key: "78636d7b5ea5532a0bdb0350168b7c99"
server_key_iv: "45575e62d41759e22a6373d9ef849cae"
server_key_version: "encryptionkey@0"
I20260810 21:00:12.122895 25085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:12.143672 25085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:12.145704 25085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:12.145821 25085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:12.146020 25085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:12.146353 25085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:12.146387 25085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:12.146432 25085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:12.146459 25085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:12.152434 25085 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:36011
I20260810 21:00:12.152534 25225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:36011 every 8 connection(s)
I20260810 21:00:12.152945 25085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:12.157351 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25085
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.161160 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.153720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:47145)
0810 21:00:12.153928 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.153966 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.154450 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.154653 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.154656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.155268 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.155529 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.155534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.156326 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.156327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.157280 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.157283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.157856 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.157857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.158019 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.158591 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.158599 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.160987 (+ 2388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
W20260810 21:00:12.161152 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.153720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:47145 (local address 127.14.200.62:43519)
0810 21:00:12.153774 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.153776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.154056 (+ 280us) server_negotiation.cc:407] Connection header received
0810 21:00:12.154533 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.154535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.154545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.154592 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.155608 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.156245 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.157382 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.157745 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.157833 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.161076 (+ 3243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:47145: BlockingRecv error: recv got EOF from 127.14.200.3:47145 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 21:00:12.161592 25226 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:43519 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.200.62:43519: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.165346 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.157999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.14.200.62:43519)
0810 21:00:12.158092 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.158094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.158211 (+ 117us) server_negotiation.cc:407] Connection header received
0810 21:00:12.158269 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.158270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.158280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.158352 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.159026 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.159586 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.160591 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.160912 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.160987 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.162332 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:12.162336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:12.162336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:12.162348 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:12.163920 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:12.164271 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:12.164274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:12.164275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:12.164375 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:12.164599 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:12.164600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:12.164600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:12.164886 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:12.165041 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:12.165133 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 21:00:12.165245 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":47,"threads_started":1}
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.166324 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.161825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:50031)
0810 21:00:12.161873 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.161883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.161928 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.162047 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.162047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.162174 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.162215 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.162216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.162991 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.162992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.163618 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.163621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.163995 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.163996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.164064 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.164447 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.164452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.166258 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:00:12.166427 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.161889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:50031 (local address 127.14.200.62:43519)
0810 21:00:12.161931 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.161932 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.161941 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:12.161981 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.161982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.161991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.162042 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.162296 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.162888 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.163717 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.163910 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.163993 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.166349 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:50031: BlockingRecv error: recv got EOF from 127.14.200.3:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.170463 25228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.166651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:51977)
0810 21:00:12.166702 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.166711 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.166747 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.166852 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.166853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.166980 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.167003 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.167004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.167584 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.167585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.168061 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.168062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.168431 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.168432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.168484 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.168900 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.168905 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.170372 (+ 1467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 21:00:12.170506 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.166722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:51977 (local address 127.14.200.62:43519)
0810 21:00:12.166763 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.166765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.166774 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:12.166795 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.166796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.166804 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.166851 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.167084 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.167509 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.168151 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.168344 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.168410 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.170437 (+ 2027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:51977: BlockingRecv error: recv got EOF from 127.14.200.3:51977 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 21:00:12.170620 25226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:12 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:12.909121 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:12.904318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:49281)
0810 21:00:12.904448 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:12.904459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:12.904510 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:12.904646 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:12.904647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:12.904837 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:12.904888 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.904890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.905766 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.905767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.906336 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:12.906338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.906664 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.906665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:12.906717 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.907122 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:12.907126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:12.909010 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260810 21:00:12.909171 25232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:12.904381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:49281 (local address 127.14.200.62:43519)
0810 21:00:12.904518 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:12.904520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:12.904536 (+ 16us) server_negotiation.cc:407] Connection header received
0810 21:00:12.904560 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:12.904561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:12.904570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:12.904631 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:12.904958 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.905676 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.906405 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:12.906598 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:12.906652 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:12.909096 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:49281: BlockingRecv error: recv got EOF from 127.14.200.1:49281 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:13.037500 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:13.032664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:45697)
0810 21:00:13.032853 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:13.032865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:13.032955 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:13.033142 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:13.033143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:13.033324 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:13.033371 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.033373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.034065 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.034066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.034603 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.034605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.035013 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.035014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.035080 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.035564 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:13.035569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:13.037382 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
W20260810 21:00:13.037545 25232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:13.032761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:45697 (local address 127.14.200.62:43519)
0810 21:00:13.032847 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:13.032849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:13.032937 (+ 88us) server_negotiation.cc:407] Connection header received
0810 21:00:13.033048 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:13.033050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:13.033063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:13.033123 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:13.033453 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.033973 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.034723 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.034928 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.035017 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.037482 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:45697: BlockingRecv error: recv got EOF from 127.14.200.2:45697 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:13.175596 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:13.171010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:54767)
0810 21:00:13.171146 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:13.171160 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:13.171232 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:13.171375 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:13.171376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:13.171584 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:13.171634 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.171636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.172303 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.172305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.172868 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.172871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.173228 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.173229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.173297 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.173712 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:13.173717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:13.175468 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
W20260810 21:00:13.175631 25232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:13.171117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:54767 (local address 127.14.200.62:43519)
0810 21:00:13.171174 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:13.171177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:13.171212 (+ 35us) server_negotiation.cc:407] Connection header received
0810 21:00:13.171301 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:13.171302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:13.171315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:13.171359 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:13.171714 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.172217 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.172958 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.173153 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.173228 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.175556 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:54767: BlockingRecv error: recv got EOF from 127.14.200.3:54767 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:13 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:13.914561 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:13.909678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:39179)
0810 21:00:13.909887 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:13.909901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:13.909983 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:13.910143 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:13.910144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:13.910310 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:13.910364 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.910366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.911108 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.911109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.911768 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:13.911771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.912250 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.912251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:13.912299 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.912722 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:13.912727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:13.914502 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
W20260810 21:00:13.914680 25236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:13.909762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:39179 (local address 127.14.200.62:43519)
0810 21:00:13.909903 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:13.909905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:13.909986 (+ 81us) server_negotiation.cc:407] Connection header received
0810 21:00:13.910053 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:13.910054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:13.910065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:13.910124 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:13.910454 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.911043 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.911881 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:13.912164 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:13.912233 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:13.914589 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:39179: BlockingRecv error: recv got EOF from 127.14.200.1:39179 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:14.042378 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:14.037983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:50997)
0810 21:00:14.038152 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:14.038163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:14.038240 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:14.038379 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:14.038380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:14.038550 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:14.038602 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.038603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.039197 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.039197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.039677 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.039678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.039968 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.039969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.040005 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.040401 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:14.040405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:14.042307 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
W20260810 21:00:14.042500 25236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:14.038064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50997 (local address 127.14.200.62:43519)
0810 21:00:14.038169 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:14.038171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:14.038223 (+ 52us) server_negotiation.cc:407] Connection header received
0810 21:00:14.038302 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:14.038303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:14.038313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:14.038360 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:14.038673 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.039137 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.039740 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.039904 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.039958 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.042419 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:50997: BlockingRecv error: recv got EOF from 127.14.200.2:50997 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:14.180868 25238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:14.176092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:55263)
0810 21:00:14.176250 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:14.176261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:14.176338 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:14.176501 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:14.176502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:14.176695 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:14.176756 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.176758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.177497 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.177498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.178033 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.178034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.178382 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.178383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.178449 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.178867 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:14.178871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:14.180761 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
W20260810 21:00:14.181030 25236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:14.176250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:55263 (local address 127.14.200.62:43519)
0810 21:00:14.176326 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:14.176328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:14.176337 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:14.176404 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:14.176405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:14.176419 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:14.176491 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:14.176897 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.177400 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.178117 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.178308 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.178382 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.180946 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:55263: BlockingRecv error: recv got EOF from 127.14.200.3:55263 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:14 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:14.919415 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:14.915032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:40887)
0810 21:00:14.915187 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:14.915198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:14.915242 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:14.915410 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:14.915411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:14.915569 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:14.915640 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.915641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.916287 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.916288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.916896 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:14.916899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.917207 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.917208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:14.917248 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.917667 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:14.917671 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:14.919350 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
W20260810 21:00:14.919565 25241 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:14.915120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:40887 (local address 127.14.200.62:43519)
0810 21:00:14.915252 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:14.915254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:14.915285 (+ 31us) server_negotiation.cc:407] Connection header received
0810 21:00:14.915306 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:14.915307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:14.915334 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:14.915393 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:14.915705 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.916207 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.916969 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:14.917147 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:14.917196 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:14.919443 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:40887: BlockingRecv error: recv got EOF from 127.14.200.1:40887 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:15.047122 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:15.042833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:55169)
0810 21:00:15.043021 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:15.043031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:15.043085 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:15.043198 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:15.043199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:15.043352 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:15.043404 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.043406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.044063 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.044064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.044545 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.044547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.044879 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.044880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.044919 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.045307 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:15.045311 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:15.047066 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":130,"threads_started":1}
W20260810 21:00:15.047252 25241 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:15.042902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:55169 (local address 127.14.200.62:43519)
0810 21:00:15.042962 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:15.042964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:15.043084 (+ 120us) server_negotiation.cc:407] Connection header received
0810 21:00:15.043131 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:15.043132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:15.043143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:15.043193 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:15.043472 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.043999 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.044610 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.044821 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.044877 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.047173 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:55169: BlockingRecv error: recv got EOF from 127.14.200.2:55169 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:15.185567 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:15.181339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:39837)
0810 21:00:15.181527 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:15.181538 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:15.181601 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:15.181719 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:15.181719 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:15.181875 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:15.181926 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.181928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.182543 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.182543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.183056 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.183058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.183351 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.183352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.183391 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.183825 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:15.183829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:15.185512 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
W20260810 21:00:15.185710 25241 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:15.181391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:39837 (local address 127.14.200.62:43519)
0810 21:00:15.181467 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:15.181469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:15.181600 (+ 131us) server_negotiation.cc:407] Connection header received
0810 21:00:15.181643 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:15.181644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:15.181655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:15.181701 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:15.181979 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.182479 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.183120 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.183292 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.183338 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.185610 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:39837: BlockingRecv error: recv got EOF from 127.14.200.3:39837 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:15 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:15.924606 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:15.919859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:36851)
0810 21:00:15.920012 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:15.920022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:15.920072 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:15.920211 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:15.920212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:15.920364 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:15.920416 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.920418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.921208 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.921210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.921725 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:15.921727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.922131 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.922132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:15.922197 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.922649 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:15.922653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:15.924490 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
W20260810 21:00:15.924644 25245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:15.919940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:36851 (local address 127.14.200.62:43519)
0810 21:00:15.920040 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:15.920042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:15.920059 (+ 17us) server_negotiation.cc:407] Connection header received
0810 21:00:15.920133 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:15.920134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:15.920143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:15.920196 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:15.920480 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.921133 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.921821 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:15.922038 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:15.922132 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:15.924574 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:36851: BlockingRecv error: recv got EOF from 127.14.200.1:36851 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:16.052233 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:16.047670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:42429)
0810 21:00:16.047894 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:16.047908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:16.047986 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:16.048172 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:16.048173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:16.048337 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:16.048390 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.048392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.049110 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.049111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.049616 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.049617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.049921 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.049922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.049969 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.050508 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:16.050512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:16.052178 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":132,"threads_started":1}
W20260810 21:00:16.052340 25245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:16.047972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:42429 (local address 127.14.200.62:43519)
0810 21:00:16.048046 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:16.048048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:16.048056 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:16.048083 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:16.048084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:16.048094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:16.048149 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:16.048466 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.049045 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.049681 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.049856 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.049902 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.052269 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:42429: BlockingRecv error: recv got EOF from 127.14.200.2:42429 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:16.190908 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:16.186158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:55605)
0810 21:00:16.186305 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:16.186323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:16.186378 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:16.186534 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:16.186535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:16.186713 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:16.186765 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.186767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.187431 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.187431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.187935 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.187937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.188253 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.188254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.188294 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.188801 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:16.188805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:16.190825 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
W20260810 21:00:16.191039 25245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:16.186223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:55605 (local address 127.14.200.62:43519)
0810 21:00:16.186307 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:16.186309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:16.186383 (+ 74us) server_negotiation.cc:407] Connection header received
0810 21:00:16.186440 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:16.186441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:16.186455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:16.186514 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:16.186873 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.187359 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.188006 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.188184 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.188239 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.190972 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:55605: BlockingRecv error: recv got EOF from 127.14.200.3:55605 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:16 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:16.929862 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:16.925184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:41297)
0810 21:00:16.925332 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:16.925346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:16.925420 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:16.925553 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:16.925554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:16.925764 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:16.925839 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.925840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.926525 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.926526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.927107 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:16.927108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.927456 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.927457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:16.927499 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.927994 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:16.927998 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:16.929801 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
W20260810 21:00:16.930044 25250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:16.925232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:41297 (local address 127.14.200.62:43519)
0810 21:00:16.925392 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:16.925394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:16.925403 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:16.925467 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:16.925468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:16.925479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:16.925535 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:16.925948 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.926459 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.927175 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:16.927381 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:16.927440 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:16.929956 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:41297: BlockingRecv error: recv got EOF from 127.14.200.1:41297 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:17.062883 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:17.052715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:39715)
0810 21:00:17.052873 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:17.052892 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:17.052958 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:17.053064 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:17.053065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:17.053230 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:17.053282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.053283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.053879 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.053880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.054398 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.054399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.054715 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.054715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.054754 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.060856 (+ 6102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:17.060861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:17.062824 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
W20260810 21:00:17.063010 25250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:17.052845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39715 (local address 127.14.200.62:43519)
0810 21:00:17.052889 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:17.052892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:17.052939 (+ 47us) server_negotiation.cc:407] Connection header received
0810 21:00:17.053002 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:17.053002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:17.053013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:17.053058 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:17.053335 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.053813 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.054462 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.054642 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.054692 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.062915 (+ 8223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:39715: BlockingRecv error: recv got EOF from 127.14.200.2:39715 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:17.195731 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:17.191370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:54409)
0810 21:00:17.191531 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:17.191545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:17.191594 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:17.191721 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:17.191722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:17.191903 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:17.191958 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.191960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.192546 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.192547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.193103 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.193106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.193388 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.193389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.193443 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.193840 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:17.193844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:17.195668 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
W20260810 21:00:17.195812 25250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:17.191423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:54409 (local address 127.14.200.62:43519)
0810 21:00:17.191509 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:17.191512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:17.191593 (+ 81us) server_negotiation.cc:407] Connection header received
0810 21:00:17.191631 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:17.191631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:17.191643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:17.191706 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:17.192017 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.192480 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.193170 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.193338 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.193377 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.195745 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:54409: BlockingRecv error: recv got EOF from 127.14.200.3:54409 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:17 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:17.934971 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:17.930277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:40339)
0810 21:00:17.930438 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:17.930448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:17.930497 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:17.930647 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:17.930647 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:17.930809 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:17.930863 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.930864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.931508 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.931509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.932033 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:17.932035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.932360 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.932361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:17.932401 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.932856 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:17.932861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:17.934876 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260810 21:00:17.935039 25255 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:17.930342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:40339 (local address 127.14.200.62:43519)
0810 21:00:17.930494 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:17.930496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:17.930504 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:17.930554 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:17.930555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:17.930564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:17.930623 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:17.930926 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.931443 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.932102 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:17.932298 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:17.932345 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:17.934952 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:40339: BlockingRecv error: recv got EOF from 127.14.200.1:40339 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:18.067833 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:18.063417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:56103)
0810 21:00:18.063555 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:18.063568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:18.063649 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:18.063815 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:18.063816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:18.063976 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:18.064036 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.064038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.064611 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.064612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.065209 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.065211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.065508 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.065508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.065549 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.066061 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:18.066066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:18.067775 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":98,"threads_started":1}
W20260810 21:00:18.067962 25255 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:18.063468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:56103 (local address 127.14.200.62:43519)
0810 21:00:18.063515 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:18.063517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:18.063650 (+ 133us) server_negotiation.cc:407] Connection header received
0810 21:00:18.063736 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:18.063736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:18.063750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:18.063801 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:18.064097 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.064546 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.065279 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.065447 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.065493 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.067859 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:56103: BlockingRecv error: recv got EOF from 127.14.200.2:56103 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:18.200520 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:18.196254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:33141)
0810 21:00:18.196406 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:18.196416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:18.196477 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:18.196613 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:18.196614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:18.196816 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:18.196875 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.196877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.197489 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.197489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.198070 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.198072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.198383 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.198383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.198423 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.198854 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:18.198858 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:18.200460 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
W20260810 21:00:18.200623 25255 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:18.196301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:33141 (local address 127.14.200.62:43519)
0810 21:00:18.196368 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:18.196371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:18.196473 (+ 102us) server_negotiation.cc:407] Connection header received
0810 21:00:18.196525 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:18.196526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:18.196537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:18.196585 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:18.196940 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.197425 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.198143 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.198318 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.198369 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.200552 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:33141: BlockingRecv error: recv got EOF from 127.14.200.3:33141 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:18 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:18.940191 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:18.935483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:34891)
0810 21:00:18.935647 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:18.935658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:18.935714 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:18.935841 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:18.935842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:18.936006 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:18.936059 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.936060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.936692 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.936693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.937224 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:18.937226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.937625 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.937626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:18.937700 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.938234 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:18.938239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:18.940095 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
W20260810 21:00:18.940249 25259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:18.935584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:34891 (local address 127.14.200.62:43519)
0810 21:00:18.935707 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:18.935709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:18.935717 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:18.935759 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:18.935759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:18.935769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:18.935825 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:18.936121 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.936627 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.937287 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:18.937541 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:18.937622 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:18.940173 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:34891: BlockingRecv error: recv got EOF from 127.14.200.1:34891 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":51,"threads_started":1}
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:19.072343 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:19.068255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:37307)
0810 21:00:19.068403 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:19.068416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:19.068476 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:19.068596 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:19.068597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:19.068756 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:19.068824 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.068826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.069418 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.069419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.069887 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.069888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.070179 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.070180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.070217 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.070626 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:19.070630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:19.072287 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260810 21:00:19.072423 25259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:19.068321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:37307 (local address 127.14.200.62:43519)
0810 21:00:19.068398 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:19.068401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:19.068474 (+ 73us) server_negotiation.cc:407] Connection header received
0810 21:00:19.068523 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:19.068524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:19.068534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:19.068577 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:19.068886 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.069348 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.069956 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.070119 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.070164 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.072376 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:37307: BlockingRecv error: recv got EOF from 127.14.200.2:37307 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:19.205902 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:19.201106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:60231)
0810 21:00:19.201283 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:19.201295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:19.201365 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:19.201522 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:19.201523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:19.201700 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:19.201760 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.201761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.202438 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.202439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.202957 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.202958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.203297 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.203298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.203378 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.203891 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:19.203896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:19.205798 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
W20260810 21:00:19.205993 25259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:19.201189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:60231 (local address 127.14.200.62:43519)
0810 21:00:19.201274 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:19.201277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:19.201345 (+ 68us) server_negotiation.cc:407] Connection header received
0810 21:00:19.201420 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:19.201421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:19.201435 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:19.201505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:19.201857 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.202351 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.203053 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.203230 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.203298 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.205920 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:60231: BlockingRecv error: recv got EOF from 127.14.200.3:60231 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:19 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:19.944705 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:19.940651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:37455)
0810 21:00:19.940837 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:19.940848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:19.940903 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:19.941042 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:19.941043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:19.941202 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:19.941254 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.941255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.941862 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.941862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.942350 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:19.942351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.942653 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.942654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:19.942691 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.943103 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:19.943107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:19.944650 (+ 1543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":81,"threads_started":1}
W20260810 21:00:19.944854 25264 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:19.940785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37455 (local address 127.14.200.62:43519)
0810 21:00:19.940906 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:19.940909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:19.940935 (+ 26us) server_negotiation.cc:407] Connection header received
0810 21:00:19.940958 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:19.940959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:19.940969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:19.941016 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:19.941320 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.941798 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.942413 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:19.942591 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:19.942636 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:19.944747 (+ 2111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:37455: BlockingRecv error: recv got EOF from 127.14.200.1:37455 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":48,"threads_started":1}
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:20.077495 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:20.072962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:48519)
0810 21:00:20.073136 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:20.073155 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:20.073211 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:20.073341 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:20.073342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:20.073507 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:20.073559 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.073560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.074257 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.074258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.074775 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.074777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.075098 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.075098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.075138 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.075577 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:20.075593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:20.077398 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
W20260810 21:00:20.077589 25264 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:20.073116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:48519 (local address 127.14.200.62:43519)
0810 21:00:20.073179 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:20.073181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:20.073193 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:00:20.073269 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:20.073270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:20.073280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:20.073319 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:20.073619 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.074197 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.074843 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.075034 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.075085 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.077498 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:48519: BlockingRecv error: recv got EOF from 127.14.200.2:48519 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:20.211009 25266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:20.206394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:55505)
0810 21:00:20.206534 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:20.206545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:20.206609 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:20.206733 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:20.206734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:20.206898 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:20.206948 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.206950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.207739 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.207740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.208246 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.208247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.208608 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.208609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.208675 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.209152 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:20.209156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:20.210940 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260810 21:00:20.211179 25264 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:20.206471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:55505 (local address 127.14.200.62:43519)
0810 21:00:20.206551 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:20.206554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:20.206604 (+ 50us) server_negotiation.cc:407] Connection header received
0810 21:00:20.206653 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:20.206654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:20.206666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:20.206715 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:20.207050 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.207641 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.208333 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.208522 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.208605 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.211057 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:55505: BlockingRecv error: recv got EOF from 127.14.200.3:55505 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:20 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:20.950069 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:20.945393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:53403)
0810 21:00:20.945544 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:20.945557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:20.945626 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:20.945750 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:20.945751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:20.945912 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:20.945975 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.945976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.946601 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.946601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.947113 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:20.947115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.947421 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.947422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:20.947460 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.947941 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:20.947946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:20.949998 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
W20260810 21:00:20.950212 25269 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:20.945485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53403 (local address 127.14.200.62:43519)
0810 21:00:20.945615 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:20.945617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:20.945625 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:20.945681 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:20.945681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:20.945690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:20.945736 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:20.946060 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.946537 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.947182 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:20.947364 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:20.947409 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:20.950108 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:53403: BlockingRecv error: recv got EOF from 127.14.200.1:53403 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:21.082728 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:21.078072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:40883)
0810 21:00:21.078241 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:21.078255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:21.078329 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:21.078494 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:21.078495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:21.078725 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:21.078812 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.078814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.079434 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.079434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.079920 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.079922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.080221 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.080221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.080266 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.080846 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:21.080850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:21.082658 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
W20260810 21:00:21.082791 25269 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:21.078199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:40883 (local address 127.14.200.62:43519)
0810 21:00:21.078249 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:21.078251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:21.078342 (+ 91us) server_negotiation.cc:407] Connection header received
0810 21:00:21.078393 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:21.078394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:21.078410 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:21.078469 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:21.078883 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.079367 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.079997 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.080162 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.080207 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.082740 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:40883: BlockingRecv error: recv got EOF from 127.14.200.2:40883 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:21.215616 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:21.211415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:34393)
0810 21:00:21.211579 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:21.211589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:21.211649 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:21.211773 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:21.211774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:21.211953 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:21.212004 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.212005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.212602 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.212603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.213114 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.213115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.213430 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.213431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.213470 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.213897 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:21.213902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:21.215562 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
W20260810 21:00:21.215724 25269 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:21.211514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:34393 (local address 127.14.200.62:43519)
0810 21:00:21.211589 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:21.211590 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:21.211633 (+ 43us) server_negotiation.cc:407] Connection header received
0810 21:00:21.211701 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:21.211702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:21.211713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:21.211755 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:21.212065 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.212540 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.213183 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.213366 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.213418 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.215652 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:34393: BlockingRecv error: recv got EOF from 127.14.200.3:34393 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:21 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:21.954787 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:21.950490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:43729)
0810 21:00:21.950660 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:21.950670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:21.950727 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:21.950858 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:21.950859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:21.951025 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:21.951090 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.951091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.951668 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.951669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.952123 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:21.952124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.952427 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.952428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:21.952466 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.952947 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:21.952952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:21.954729 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
W20260810 21:00:21.954936 25273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:21.950556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:43729 (local address 127.14.200.62:43519)
0810 21:00:21.950678 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:21.950680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:21.950713 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:00:21.950782 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:21.950783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:21.950792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:21.950838 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:21.951147 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.951609 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.952200 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:21.952370 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:21.952415 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:21.954837 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:43729: BlockingRecv error: recv got EOF from 127.14.200.1:43729 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:22.093062 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:22.083406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:60091)
0810 21:00:22.083566 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:22.083576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:22.083632 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:22.083769 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:22.083769 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:22.083935 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:22.083989 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.083990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.084619 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.084620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.085220 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.085222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.085552 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.085552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.085593 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.091136 (+ 5543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:22.091140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:22.092992 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
W20260810 21:00:22.093227 25273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:22.083481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60091 (local address 127.14.200.62:43519)
0810 21:00:22.083548 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:22.083550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:22.083634 (+ 84us) server_negotiation.cc:407] Connection header received
0810 21:00:22.083693 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:22.083694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:22.083705 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:22.083745 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:22.084047 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.084553 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.085287 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.085470 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.085520 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.093091 (+ 7571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:60091: BlockingRecv error: recv got EOF from 127.14.200.2:60091 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:22.221133 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:22.216248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:59471)
0810 21:00:22.216392 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:22.216419 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:22.216485 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:22.216670 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:22.216671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:22.216911 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:22.216961 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.216963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.217665 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.217666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.218206 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.218208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.218569 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.218570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.218632 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.219144 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:22.219152 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:22.221027 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
W20260810 21:00:22.221163 25273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:22.216380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:59471 (local address 127.14.200.62:43519)
0810 21:00:22.216449 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:22.216451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:22.216464 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:00:22.216564 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:22.216565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:22.216579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:22.216660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:22.217070 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.217570 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.218289 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.218477 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.218567 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.221098 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:59471: BlockingRecv error: recv got EOF from 127.14.200.3:59471 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:22 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:22.960012 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:22.955202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:49681)
0810 21:00:22.955332 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:22.955369 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:22.955429 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:22.955617 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:22.955618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:22.955838 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:22.955890 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.955891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.956698 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.956699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.957419 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:22.957421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.957743 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.957743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:22.957786 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.958184 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:22.958188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:22.959945 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260810 21:00:22.960105 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:22.955259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:49681 (local address 127.14.200.62:43519)
0810 21:00:22.955423 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:22.955425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:22.955435 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:00:22.955526 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:22.955527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:22.955536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:22.955600 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:22.955996 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.956629 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.957493 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:22.957681 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:22.957732 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:22.960031 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:49681: BlockingRecv error: recv got EOF from 127.14.200.1:49681 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:23.097955 25279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:23.093477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:50995)
0810 21:00:23.093661 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:23.093670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:23.093718 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:23.093841 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:23.093842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:23.094028 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:23.094076 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.094077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.094688 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.094688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.095175 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.095177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.095464 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.095464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.095503 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.095918 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:23.095922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:23.097831 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
W20260810 21:00:23.097998 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:23.093564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50995 (local address 127.14.200.62:43519)
0810 21:00:23.093619 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:23.093621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:23.093719 (+ 98us) server_negotiation.cc:407] Connection header received
0810 21:00:23.093772 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:23.093773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:23.093783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:23.093821 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:23.094133 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.094626 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.095238 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.095405 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.095450 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.097934 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:50995: BlockingRecv error: recv got EOF from 127.14.200.2:50995 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:23.225993 25280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:23.221555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:38495)
0810 21:00:23.221705 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:23.221714 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:23.221764 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:23.221866 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:23.221866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:23.222025 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:23.222072 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.222074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.222734 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.222735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.223275 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.223277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.223629 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.223630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.223671 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.224107 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:23.224113 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:23.225866 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260810 21:00:23.226079 25278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:23.221607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38495 (local address 127.14.200.62:43519)
0810 21:00:23.221646 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:23.221650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:23.221763 (+ 113us) server_negotiation.cc:407] Connection header received
0810 21:00:23.221805 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:23.221806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:23.221817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:23.221862 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:23.222121 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.222616 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.223332 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.223537 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.223586 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.225997 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:38495: BlockingRecv error: recv got EOF from 127.14.200.3:38495 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:23 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:23.965118 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:23.960479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:33167)
0810 21:00:23.960622 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:23.960631 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:23.960681 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:23.960849 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:23.960850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:23.961053 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:23.961106 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.961107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.961709 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.961710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.962202 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:23.962203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.962552 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.962553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:23.962595 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.963032 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:23.963036 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:23.965026 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
W20260810 21:00:23.965176 25283 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:23.960527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33167 (local address 127.14.200.62:43519)
0810 21:00:23.960673 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:23.960674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:23.960683 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:23.960751 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:23.960752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:23.960762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:23.960840 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:23.961169 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.961645 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.962266 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:23.962454 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:23.962524 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:23.965105 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:33167: BlockingRecv error: recv got EOF from 127.14.200.1:33167 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:24.103018 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:24.098568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:39661)
0810 21:00:24.098744 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:24.098754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:24.098814 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:24.098939 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:24.098939 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:24.099100 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:24.099149 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.099150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.099745 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.099745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.100246 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.100247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.100588 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.100588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.100630 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.101098 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:24.101102 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:24.102922 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
W20260810 21:00:24.103071 25283 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:24.098638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:39661 (local address 127.14.200.62:43519)
0810 21:00:24.098706 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:24.098708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:24.098815 (+ 107us) server_negotiation.cc:407] Connection header received
0810 21:00:24.098869 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:24.098870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:24.098881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:24.098918 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:24.099205 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.099679 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.100311 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.100503 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.100575 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.102995 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:39661: BlockingRecv error: recv got EOF from 127.14.200.2:39661 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:24.230899 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:24.226434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:49417)
0810 21:00:24.226592 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:24.226610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:24.226667 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:24.226793 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:24.226794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:24.226966 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:24.227013 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.227015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.227648 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.227649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.228125 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.228126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.228414 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.228415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.228456 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.229009 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:24.229013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:24.230841 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
W20260810 21:00:24.231038 25283 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:24.226485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:49417 (local address 127.14.200.62:43519)
0810 21:00:24.226556 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:24.226558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:24.226670 (+ 112us) server_negotiation.cc:407] Connection header received
0810 21:00:24.226711 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:24.226711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:24.226723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:24.226773 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:24.227077 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.227590 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.228188 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.228348 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.228393 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.230954 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:49417: BlockingRecv error: recv got EOF from 127.14.200.3:49417 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:24 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:24.969870 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:24.965594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:51489)
0810 21:00:24.965763 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:24.965778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:24.965862 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:24.965992 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:24.965993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:24.966159 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:24.966209 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.966210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.966847 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.966848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.967368 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:24.967369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.967696 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.967697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:24.967737 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.968173 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:24.968177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:24.969812 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
W20260810 21:00:24.970009 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:24.965737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:51489 (local address 127.14.200.62:43519)
0810 21:00:24.965884 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:24.965886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:24.965893 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:24.965915 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:24.965915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:24.965924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:24.965978 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:24.966265 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.966778 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.967437 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:24.967634 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:24.967682 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:24.969922 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:51489: BlockingRecv error: recv got EOF from 127.14.200.1:51489 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:25.107745 25288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:25.103454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:48409)
0810 21:00:25.103595 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:25.103607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:25.103664 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:25.103806 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:25.103807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:25.103993 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:25.104060 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.104062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.104659 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.104660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.105202 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.105203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.105540 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.105541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.105579 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.105974 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:25.105978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:25.107678 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260810 21:00:25.107846 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:25.103578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:48409 (local address 127.14.200.62:43519)
0810 21:00:25.103647 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:25.103649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:25.103657 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:25.103732 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:25.103732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:25.103743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:25.103785 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:25.104125 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.104594 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.105278 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.105461 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.105515 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.107773 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:48409: BlockingRecv error: recv got EOF from 127.14.200.2:48409 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:25.236245 25289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:25.231393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:54223)
0810 21:00:25.231578 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:25.231590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:25.231677 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:25.231824 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:25.231826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:25.232028 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:25.232074 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.232076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.232857 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.232858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.233411 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.233413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.233793 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.233794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.233855 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.234373 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:25.234378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:25.236179 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
W20260810 21:00:25.236351 25287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:25.231463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:54223 (local address 127.14.200.62:43519)
0810 21:00:25.231541 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:25.231544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:25.231657 (+ 113us) server_negotiation.cc:407] Connection header received
0810 21:00:25.231731 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:25.231732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:25.231746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:25.231817 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:25.232157 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.232755 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.233510 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.233706 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.233792 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.236278 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:54223: BlockingRecv error: recv got EOF from 127.14.200.3:54223 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:25 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:25.978832 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:25.970513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:50651)
0810 21:00:25.970679 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:25.970703 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:25.970839 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:25.971064 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:25.971065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:25.971395 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:25.971508 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.971511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.972573 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.972573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.973686 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:25.973689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.974265 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.974266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:25.974351 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.975223 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:25.975236 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:25.978715 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260810 21:00:25.978928 25291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:25.970612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:50651 (local address 127.14.200.62:43519)
0810 21:00:25.970834 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:25.970836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:25.970845 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:25.970933 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:25.970937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:25.970957 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:25.971047 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:25.971622 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.972443 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.973836 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:25.974187 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:25.974262 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:25.978847 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:50651: BlockingRecv error: recv got EOF from 127.14.200.1:50651 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":143,"threads_started":1}
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:26.116501 25292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:26.108322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:44715)
0810 21:00:26.108504 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:26.108537 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:26.108618 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:26.108844 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:26.108845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:26.109204 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:26.109299 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.109309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.110705 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.110707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.111513 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.111516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.112138 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.112139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.112239 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.112998 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:26.113009 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:26.116305 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
W20260810 21:00:26.116587 25291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:26.108451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:44715 (local address 127.14.200.62:43519)
0810 21:00:26.108534 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:26.108536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:26.108604 (+ 68us) server_negotiation.cc:407] Connection header received
0810 21:00:26.108702 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:26.108703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:26.108715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:26.108830 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:26.109422 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.110531 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.111677 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.111986 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.112134 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.116452 (+ 4318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:44715: BlockingRecv error: recv got EOF from 127.14.200.2:44715 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.3@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:26.244616 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:26.236968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.3:49477)
0810 21:00:26.237231 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:26.237250 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:26.237365 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:26.237594 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:26.237595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:26.237964 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:26.238072 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.238074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.239092 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.239092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.239910 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.239912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.240441 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.240442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.240478 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.241248 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:26.241252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:26.244536 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":172,"threads_started":1}
W20260810 21:00:26.244740 25291 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:26.237046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:49477 (local address 127.14.200.62:43519)
0810 21:00:26.237154 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:26.237166 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:26.237362 (+ 196us) server_negotiation.cc:407] Connection header received
0810 21:00:26.237462 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:26.237462 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:26.237476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:26.237565 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:26.238172 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.239010 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.240021 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.240346 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.240440 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.244648 (+ 4208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.3:49477: BlockingRecv error: recv got EOF from 127.14.200.3:49477 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:26 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.1@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:26.986927 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:26.979598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.1:59117)
0810 21:00:26.979757 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:26.979768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:26.979827 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:26.980064 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:26.980065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:26.980343 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:26.980434 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.980436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.981531 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.981532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.982387 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:26.982389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.983004 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.983005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:26.983128 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.983685 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:26.983698 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:26.986802 (+ 3104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
W20260810 21:00:26.987085 25295 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:26.979700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:59117 (local address 127.14.200.62:43519)
0810 21:00:26.979833 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:26.979837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:26.979848 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:00:26.979891 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:26.979892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:26.979911 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:26.980038 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:26.980541 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.981363 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.982530 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:26.982870 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:26.982973 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:26.986981 (+ 4008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.1:59117: BlockingRecv error: recv got EOF from 127.14.200.1:59117 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
Aug 10 21:00:27 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 21:00:27 dist-test-slave-1x4g krb5kdc[24139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.200.2@KRBTEST.COM for oryx/127.14.200.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 21:00:27.133396 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:27.117310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:43519 (local address 127.14.200.2:54423)
0810 21:00:27.117646 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:27.117657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:27.117754 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:27.117992 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:27.117993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:27.118274 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:27.118370 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:27.118379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.119339 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.119339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:27.120120 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:27.120122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.120588 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.120589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:27.120644 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:27.130218 (+ 9574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:27.130223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:27.133227 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.200.62:43519: Server oryx/127.14.200.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":169,"threads_started":1}
W20260810 21:00:27.133471 25295 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:27.117438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54423 (local address 127.14.200.62:43519)
0810 21:00:27.117565 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:27.117567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:27.117755 (+ 188us) server_negotiation.cc:407] Connection header received
0810 21:00:27.117874 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:27.117875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:27.117896 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:27.117974 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:27.118498 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.119244 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.120215 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.120465 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.120566 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:27.133366 (+ 12800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.200.2:54423: BlockingRecv error: recv got EOF from 127.14.200.2:54423 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 21:00:27.159878 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24781
I20260810 21:00:27.172700 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24940
I20260810 21:00:27.182298 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25085
I20260810 21:00:27.191934 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 24704
2026-08-10T21:00:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17598 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T21:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:27Z Disabled control of system clock
I20260810 21:00:27.235128 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:40129
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:40129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 21:00:27.372186 25301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:27.372535 25301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:27.372575 25301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:27.374307 25301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:27.374363 25301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:27.374388 25301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:27.374408 25301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:27.374432 25301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:27.376237 25301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:40129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:40129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:27.376555 25301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:27.376991 25301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:27.385095 25306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:27.385155 25309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:27.385221 25301 server_base.cc:1034] running on GCE node
W20260810 21:00:27.385177 25307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:27.385754 25301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:27.386222 25301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:27.387415 25301 hybrid_clock.cc:648] HybridClock initialized: now 1786395627387388 us; error 67 us; skew 500 ppm
I20260810 21:00:27.389442 25301 webserver.cc:460] Webserver started at http://127.14.200.62:39059/ using document root <none> and password file <none>
I20260810 21:00:27.389881 25301 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:27.389953 25301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:27.390254 25301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:27.391891 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "2d09e565f3c54ffd83ff68522c9fe011"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "f74f8c03feba586c5ef348d49f6cb068"
server_key_iv: "21677be0bb035166f7924cfdd451fc32"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.392522 25301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "2d09e565f3c54ffd83ff68522c9fe011"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "f74f8c03feba586c5ef348d49f6cb068"
server_key_iv: "21677be0bb035166f7924cfdd451fc32"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.395102 25301 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260810 21:00:27.396703 25317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:27.397071 25301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:27.397295 25301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "2d09e565f3c54ffd83ff68522c9fe011"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "f74f8c03feba586c5ef348d49f6cb068"
server_key_iv: "21677be0bb035166f7924cfdd451fc32"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.397426 25301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:27.457921 25301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:27.458504 25301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:27.458746 25301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:27.465792 25301 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:40129
I20260810 21:00:27.465881 25374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:40129 every 8 connection(s)
I20260810 21:00:27.466475 25301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:27.467696 25375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:27.472483 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011: Bootstrap starting.
I20260810 21:00:27.473459 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25301
I20260810 21:00:27.473664 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:27.473754 25375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:27.474052 15136 external_mini_cluster.cc:1428] Setting key dd65a629d490724674d962feb5469a42
I20260810 21:00:27.474375 25375 log.cc:824] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:27.475695 25375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011: No bootstrap required, opened a new log
I20260810 21:00:27.478600 25375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } }
I20260810 21:00:27.478981 25375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:27.479058 25375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d09e565f3c54ffd83ff68522c9fe011, State: Initialized, Role: FOLLOWER
I20260810 21:00:27.479333 25375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } }
I20260810 21:00:27.480055 25375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:27.480207 25375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:27.480352 25375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:27.481550 25375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } }
I20260810 21:00:27.481805 25375 leader_election.cc:302] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d09e565f3c54ffd83ff68522c9fe011; no voters:
I20260810 21:00:27.482160 25375 leader_election.cc:288] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:27.482230 25380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:27.482577 25380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [term 1 LEADER]: Becoming Leader. State: Replica: 2d09e565f3c54ffd83ff68522c9fe011, State: Running, Role: LEADER
I20260810 21:00:27.482688 25375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:27.482853 25380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } }
I20260810 21:00:27.483953 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:27.474837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.14.200.62:40129)
0810 21:00:27.475194 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:27.475221 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:27.475266 (+ 45us) server_negotiation.cc:407] Connection header received
0810 21:00:27.476076 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:27.476083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:27.476185 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:27.476371 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:27.476889 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.478316 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.480182 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.481128 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.481279 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:27.482426 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:27.482445 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:27.482445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:27.482837 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:27.482924 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:27.482954 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:27.483061 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:27.483296 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 21:00:27.483460 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":141,"threads_started":1}
I20260810 21:00:27.484052 25381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d09e565f3c54ffd83ff68522c9fe011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } } }
I20260810 21:00:27.484365 25382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d09e565f3c54ffd83ff68522c9fe011. Latest consensus state: current_term: 1 leader_uuid: "2d09e565f3c54ffd83ff68522c9fe011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d09e565f3c54ffd83ff68522c9fe011" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 40129 } } }
I20260810 21:00:27.484537 25381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:27.484576 25382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:27.485399 25390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:27.486673 25390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:27.489959 25390 catalog_manager.cc:1347] Generated new cluster ID: e60780094e4d47e8bea7952f09708bd2
I20260810 21:00:27.490092 25390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:00:27.523639 25390 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:27.524726 25390 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:27.537907 25390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d09e565f3c54ffd83ff68522c9fe011: Generated new TSK 0
I20260810 21:00:27.538260 25390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:00:27.547336 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--builtin_ntp_servers=127.14.200.20:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 21:00:27.701215 25400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:27.701555 25400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:27.701591 25400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:27.703615 25400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:27.703702 25400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:27.703740 25400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:27.706001 25400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:27.706439 25400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:27.706940 25400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:27.715258 25406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:27.715276 25408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:27.715327 25410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:27.715705 25400 server_base.cc:1034] running on GCE node
I20260810 21:00:27.716051 25400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:27.716485 25400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:27.717747 25400 hybrid_clock.cc:648] HybridClock initialized: now 1786395627717706 us; error 69 us; skew 500 ppm
I20260810 21:00:27.720330 25400 webserver.cc:460] Webserver started at http://127.14.200.1:33773/ using document root <none> and password file <none>
I20260810 21:00:27.721055 25400 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:27.721225 25400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:27.721535 25400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:27.724332 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "5a8dee3f8471498682c8ac788f05ec23"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "0f50458516814485a9a68c0ecc1a5acd"
server_key_iv: "5b7501a8280eaa1af9f61359097ca52c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.725248 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "5a8dee3f8471498682c8ac788f05ec23"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "0f50458516814485a9a68c0ecc1a5acd"
server_key_iv: "5b7501a8280eaa1af9f61359097ca52c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.728493 25400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 21:00:27.730333 25416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:27.730657 25400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:27.730859 25400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "5a8dee3f8471498682c8ac788f05ec23"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "0f50458516814485a9a68c0ecc1a5acd"
server_key_iv: "5b7501a8280eaa1af9f61359097ca52c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.731005 25400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:27.759862 25400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:27.760546 25400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:27.760952 25400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:27.761435 25400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:27.762107 25400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:27.762209 25400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:27.762329 25400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:27.762404 25400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:27.775250 25400 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42189
I20260810 21:00:27.775327 25539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42189 every 8 connection(s)
I20260810 21:00:27.776044 25400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:27.779557 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25400
I20260810 21:00:27.779699 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:27.780074 15136 external_mini_cluster.cc:1428] Setting key 257a6faf3cab6eaf838ca624e63070e7
I20260810 21:00:27.781879 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--builtin_ntp_servers=127.14.200.20:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 21:00:27.786339 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:27.777386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33023 (local address 127.14.200.62:40129)
0810 21:00:27.777489 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:27.777491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:27.777951 (+ 460us) server_negotiation.cc:407] Connection header received
0810 21:00:27.778991 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:27.778993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:27.779033 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:27.779135 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:27.780407 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.781378 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.783315 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.783644 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.783903 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:27.785821 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:27.785822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:27.785823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:27.785833 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:27.785868 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:27.785884 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:27.786024 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:27.786125 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 21:00:27.786164 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 21:00:27.786743 25542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:27.777433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:40129 (local address 127.14.200.1:33023)
0810 21:00:27.777747 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:27.777814 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:27.778797 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:27.779268 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:27.779277 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:27.779290 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:27.780258 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:27.780265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.781567 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.781568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:27.783084 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:27.783095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:27.783908 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:27.783909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:27.784452 (+ 543us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:27.785545 (+ 1093us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:27.785546 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:27.785610 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:27.785656 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:27.786016 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:27.786020 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:27.786022 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:27.786124 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 21:00:27.786263 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":138,"threads_started":1}
I20260810 21:00:27.787680 25540 heartbeater.cc:347] Connected to a master server at 127.14.200.62:40129
I20260810 21:00:27.787832 25540 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:27.788240 25540 heartbeater.cc:511] Master 127.14.200.62:40129 requested a full tablet report, sending...
I20260810 21:00:27.789389 25335 ts_manager.cc:194] Registered new tserver with Master: 5a8dee3f8471498682c8ac788f05ec23 (127.14.200.1:42189)
I20260810 21:00:27.790446 25335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.1:33023
W20260810 21:00:27.965380 25543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:27.965832 25543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:27.965902 25543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:27.967783 25543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:27.967885 25543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:27.967953 25543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:27.970269 25543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:27.970758 25543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:27.971367 25543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:27.980147 25549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:27.980134 25553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:27.980234 25550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:27.980917 25543 server_base.cc:1034] running on GCE node
I20260810 21:00:27.981298 25543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:27.981717 25543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:27.982935 25543 hybrid_clock.cc:648] HybridClock initialized: now 1786395627982913 us; error 53 us; skew 500 ppm
I20260810 21:00:27.985671 25543 webserver.cc:460] Webserver started at http://127.14.200.2:45785/ using document root <none> and password file <none>
I20260810 21:00:27.986088 25543 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:27.986238 25543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:27.986509 25543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:27.988281 25543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "ddb266d85c064e2b9ceb18775356d4d2"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "ec07c6aea6ccffaf1a650672f243aa92"
server_key_iv: "e03b017e980e1a242f9d3c0681f1378c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.989126 25543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "ddb266d85c064e2b9ceb18775356d4d2"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "ec07c6aea6ccffaf1a650672f243aa92"
server_key_iv: "e03b017e980e1a242f9d3c0681f1378c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.991768 25543 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 21:00:27.993115 25561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:27.993404 25543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:27.993597 25543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "ddb266d85c064e2b9ceb18775356d4d2"
format_stamp: "Formatted at 2026-08-10 21:00:27 on dist-test-slave-1x4g"
server_key: "ec07c6aea6ccffaf1a650672f243aa92"
server_key_iv: "e03b017e980e1a242f9d3c0681f1378c"
server_key_version: "encryptionkey@0"
I20260810 21:00:27.993712 25543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.019234 25543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.019542 25543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.019709 25543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.019974 25543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:28.020459 25543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:28.020526 25543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.020561 25543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:28.020605 25543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.026618 25543 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:39117
I20260810 21:00:28.026669 25686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:39117 every 8 connection(s)
I20260810 21:00:28.026976 25543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:00:28.032006 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.027667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:59671 (local address 127.14.200.62:40129)
0810 21:00:28.027751 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.027753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.027982 (+ 229us) server_negotiation.cc:407] Connection header received
0810 21:00:28.028503 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.028504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.028517 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.028571 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.029208 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.029694 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.030633 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.030847 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.030938 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:28.031731 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.031732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.031733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.031737 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.031750 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.031754 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.031800 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.031896 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.031921 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 21:00:28.032207 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.027649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:40129 (local address 127.14.200.2:59671)
0810 21:00:28.027862 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:28.027894 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:28.028381 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:28.028638 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:28.028641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:28.028649 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:28.029112 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:28.029117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.029790 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.029791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:28.030541 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:28.030543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.030952 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.030953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:28.031073 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:28.031580 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:28.031580 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:28.031607 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:28.031627 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:28.031799 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:28.031804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:28.031807 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:28.031889 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 21:00:28.031985 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260810 21:00:28.032678 25687 heartbeater.cc:347] Connected to a master server at 127.14.200.62:40129
I20260810 21:00:28.032819 25687 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:28.033012 25687 heartbeater.cc:511] Master 127.14.200.62:40129 requested a full tablet report, sending...
I20260810 21:00:28.033384 25335 ts_manager.cc:194] Registered new tserver with Master: ddb266d85c064e2b9ceb18775356d4d2 (127.14.200.2:39117)
I20260810 21:00:28.033712 25335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.2:59671
I20260810 21:00:28.034075 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25543
I20260810 21:00:28.034139 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:28.034307 15136 external_mini_cluster.cc:1428] Setting key c62dec848ce6d585304f2c58d86980b8
I20260810 21:00:28.035133 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--builtin_ntp_servers=127.14.200.20:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 21:00:28.115604 25690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:28.115813 25690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:28.115845 25690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:28.117061 25690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:28.117116 25690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:28.117158 25690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:28.118474 25690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:40129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:28.118721 25690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:28.119032 25690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:28.123874 25696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.123924 25698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.123885 25695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.124146 25690 server_base.cc:1034] running on GCE node
I20260810 21:00:28.124370 25690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:28.124605 25690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:28.125741 25690 hybrid_clock.cc:648] HybridClock initialized: now 1786395628125728 us; error 29 us; skew 500 ppm
I20260810 21:00:28.127024 25690 webserver.cc:460] Webserver started at http://127.14.200.3:36281/ using document root <none> and password file <none>
I20260810 21:00:28.127270 25690 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:28.127317 25690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:28.127489 25690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:28.128548 25690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "9a9266e353be437f97d28deff04d5fbf"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9308c3e013142da0477e3d11352bc693"
server_key_iv: "c48eef55c7138918839fbc9c1d83d583"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.128926 25690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "9a9266e353be437f97d28deff04d5fbf"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9308c3e013142da0477e3d11352bc693"
server_key_iv: "c48eef55c7138918839fbc9c1d83d583"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.130311 25690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:28.131034 25704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.131204 25690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:28.131310 25690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "9a9266e353be437f97d28deff04d5fbf"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9308c3e013142da0477e3d11352bc693"
server_key_iv: "c48eef55c7138918839fbc9c1d83d583"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.131384 25690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.148669 25690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.149121 25690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.149276 25690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.149497 25690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:28.149845 25690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:28.149899 25690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.149941 25690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:28.149971 25690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.155926 25690 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:46851
I20260810 21:00:28.155975 25826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:46851 every 8 connection(s)
I20260810 21:00:28.156296 25690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:28.160740 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25690
I20260810 21:00:28.160868 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:28.161113 15136 external_mini_cluster.cc:1428] Setting key b922e9ca393e078a6d54173b1f01ecb9
I20260810 21:00:28.161576 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.157132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:50179 (local address 127.14.200.62:40129)
0810 21:00:28.157202 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.157204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.157448 (+ 244us) server_negotiation.cc:407] Connection header received
0810 21:00:28.157927 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.157927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.157938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.157991 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.158660 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.159144 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.160050 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.160223 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.160270 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:28.161177 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.161179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.161179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.161184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.161203 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.161208 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.161262 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.161468 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.161499 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 21:00:28.161777 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.157118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:40129 (local address 127.14.200.3:50179)
0810 21:00:28.157324 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:28.157357 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:28.157843 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:28.158045 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:28.158049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:28.158057 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:28.158583 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:28.158588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.159224 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.159225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:28.159974 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:28.159976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:28.160297 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:28.160298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:28.160399 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:28.160984 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:28.160985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:28.161029 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:28.161058 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:28.161378 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:28.161385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:28.161387 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:28.161466 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 21:00:28.161575 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260810 21:00:28.162174 25827 heartbeater.cc:347] Connected to a master server at 127.14.200.62:40129
I20260810 21:00:28.162274 25827 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:28.162474 25827 heartbeater.cc:511] Master 127.14.200.62:40129 requested a full tablet report, sending...
I20260810 21:00:28.162835 25335 ts_manager.cc:194] Registered new tserver with Master: 9a9266e353be437f97d28deff04d5fbf (127.14.200.3:46851)
I20260810 21:00:28.163206 25335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.3:50179
I20260810 21:00:28.173166 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 21:00:28.174971 25379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:28.174507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.14.200.62:40129)
0810 21:00:28.174560 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.174562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.174695 (+ 133us) server_negotiation.cc:407] Connection header received
0810 21:00:28.174746 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.174747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.174755 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.174796 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.174903 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53178: BlockingRecv error: recv got EOF from 127.0.0.1:53178 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:00:28.175082 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25400
I20260810 21:00:28.182308 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25543
I20260810 21:00:28.189626 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25690
I20260810 21:00:28.195017 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25301
2026-08-10T21:00:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (993 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T21:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:28Z Disabled control of system clock
I20260810 21:00:28.223024 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:34529
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 21:00:28.303146 25838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:28.303331 25838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:28.303352 25838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:28.304375 25838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:28.304406 25838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:28.304418 25838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:28.304430 25838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:28.304442 25838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:28.305724 25838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:34529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:28.305902 25838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:28.306126 25838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:28.310828 25843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.310909 25844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.310856 25846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.310981 25838 server_base.cc:1034] running on GCE node
I20260810 21:00:28.311295 25838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:28.311592 25838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:28.312731 25838 hybrid_clock.cc:648] HybridClock initialized: now 1786395628312719 us; error 25 us; skew 500 ppm
I20260810 21:00:28.313905 25838 webserver.cc:460] Webserver started at http://127.14.200.62:39199/ using document root <none> and password file <none>
I20260810 21:00:28.314170 25838 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:28.314244 25838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:28.314386 25838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:28.315610 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "1f763eec69104ea9b8871b6322570434"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9c9642889ea045dfb6489ce7549301c8"
server_key_iv: "b005a2ada1deca0b37435ee834a0af99"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.315917 25838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "1f763eec69104ea9b8871b6322570434"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9c9642889ea045dfb6489ce7549301c8"
server_key_iv: "b005a2ada1deca0b37435ee834a0af99"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.317229 25838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:00:28.317945 25852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.318169 25838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:28.318265 25838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "1f763eec69104ea9b8871b6322570434"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "9c9642889ea045dfb6489ce7549301c8"
server_key_iv: "b005a2ada1deca0b37435ee834a0af99"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.318339 25838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.321661 25838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.321941 25838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.322077 25838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.325970 25838 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:34529
I20260810 21:00:28.325999 25910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:34529 every 8 connection(s)
I20260810 21:00:28.326313 25838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:28.326956 25911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:28.327180 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25838
I20260810 21:00:28.327278 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:28.327492 15136 external_mini_cluster.cc:1428] Setting key b6bc68a2b48a6ff59c62b6cd7eb92be2
I20260810 21:00:28.330013 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.328054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.14.200.62:34529)
0810 21:00:28.328245 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.328251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.328274 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:00:28.328669 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.328676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.328712 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.328913 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.329564 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.329567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.329567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.329571 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.329609 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.329618 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.329675 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.329767 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.329835 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260810 21:00:28.330832 25911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434: Bootstrap starting.
I20260810 21:00:28.331389 25911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:28.331668 25911 log.cc:824] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:28.332320 25911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434: No bootstrap required, opened a new log
I20260810 21:00:28.333658 25911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } }
I20260810 21:00:28.333804 25911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:28.333873 25911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f763eec69104ea9b8871b6322570434, State: Initialized, Role: FOLLOWER
I20260810 21:00:28.334012 25911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } }
I20260810 21:00:28.334093 25911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:28.334136 25911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:28.334182 25911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:28.334692 25911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } }
I20260810 21:00:28.334815 25911 leader_election.cc:302] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f763eec69104ea9b8871b6322570434; no voters:
I20260810 21:00:28.334983 25911 leader_election.cc:288] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:28.335242 25911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:28.335273 25917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:28.335456 25917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [term 1 LEADER]: Becoming Leader. State: Replica: 1f763eec69104ea9b8871b6322570434, State: Running, Role: LEADER
I20260810 21:00:28.335559 25917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } }
I20260810 21:00:28.335884 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f763eec69104ea9b8871b6322570434. Latest consensus state: current_term: 1 leader_uuid: "1f763eec69104ea9b8871b6322570434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } } }
I20260810 21:00:28.335896 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f763eec69104ea9b8871b6322570434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f763eec69104ea9b8871b6322570434" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34529 } } }
I20260810 21:00:28.336015 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:28.336043 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:28.336223 25928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:28.336736 25928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:28.338407 25928 catalog_manager.cc:1347] Generated new cluster ID: a79bc7ddd62548788d4927ed368104c4
I20260810 21:00:28.338466 25928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:00:28.348498 25928 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:28.348680 25928 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:28.354445 25928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f763eec69104ea9b8871b6322570434: Generated new TSK 0
I20260810 21:00:28.354625 25928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:00:28.358211 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--builtin_ntp_servers=127.14.200.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 21:00:28.442974 25938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:28.443178 25938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:28.443213 25938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:28.444319 25938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:28.444382 25938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:28.444425 25938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:28.445755 25938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:28.446015 25938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:28.446296 25938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:28.452368 25944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.452471 25938 server_base.cc:1034] running on GCE node
W20260810 21:00:28.452546 25943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.452672 25948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.452906 25938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:28.453177 25938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:28.454341 25938 hybrid_clock.cc:648] HybridClock initialized: now 1786395628454318 us; error 38 us; skew 500 ppm
I20260810 21:00:28.455648 25938 webserver.cc:460] Webserver started at http://127.14.200.1:43373/ using document root <none> and password file <none>
I20260810 21:00:28.455883 25938 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:28.455952 25938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:28.456092 25938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:28.457232 25938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "0071e27592c04f309a5ebb5e9bee3f14"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "13ea05ca68eba4bda86a6f76c112d14a"
server_key_iv: "5dbda15c1c30009af4181dbdb0cc4dcd"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.457616 25938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "0071e27592c04f309a5ebb5e9bee3f14"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "13ea05ca68eba4bda86a6f76c112d14a"
server_key_iv: "5dbda15c1c30009af4181dbdb0cc4dcd"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.458941 25938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:28.459668 25954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.459828 25938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:28.459929 25938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "0071e27592c04f309a5ebb5e9bee3f14"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "13ea05ca68eba4bda86a6f76c112d14a"
server_key_iv: "5dbda15c1c30009af4181dbdb0cc4dcd"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.460001 25938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.463253 25938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.463515 25938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.463649 25938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.463850 25938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:28.464151 25938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:28.464203 25938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.464239 25938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:28.464267 25938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.470165 25938 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46415
I20260810 21:00:28.470199 26074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46415 every 8 connection(s)
I20260810 21:00:28.470501 25938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:00:28.473348 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.471209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:34677 (local address 127.14.200.62:34529)
0810 21:00:28.471269 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.471271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.471482 (+ 211us) server_negotiation.cc:407] Connection header received
0810 21:00:28.471941 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.471942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.471948 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.471998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.473175 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.473176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.473176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.473179 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.473190 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.473193 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.473235 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.473287 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.473298 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:00:28.473552 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.471182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34529 (local address 127.14.200.1:34677)
0810 21:00:28.471361 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:28.471390 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:28.471829 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:28.472048 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:28.472051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:28.472059 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:28.473051 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:28.473051 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:28.473077 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:28.473104 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:28.473234 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:28.473236 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:28.473238 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:28.473285 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 21:00:28.473327 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260810 21:00:28.474110 26075 heartbeater.cc:347] Connected to a master server at 127.14.200.62:34529
I20260810 21:00:28.474215 26075 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:28.474366 26075 heartbeater.cc:511] Master 127.14.200.62:34529 requested a full tablet report, sending...
I20260810 21:00:28.474833 25870 ts_manager.cc:194] Registered new tserver with Master: 0071e27592c04f309a5ebb5e9bee3f14 (127.14.200.1:46415)
I20260810 21:00:28.477716 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 25938
I20260810 21:00:28.477826 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:28.478029 15136 external_mini_cluster.cc:1428] Setting key 39c02fe042c18e978240455ceb38fb60
I20260810 21:00:28.478905 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--builtin_ntp_servers=127.14.200.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 21:00:28.557864 26078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:28.558050 26078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:28.558070 26078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:28.559168 26078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:28.559207 26078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:28.559257 26078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:28.560880 26078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:28.561112 26078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:28.561390 26078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:28.566170 26086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.566215 26085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.566157 26088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.566366 26078 server_base.cc:1034] running on GCE node
I20260810 21:00:28.566586 26078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:28.566835 26078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:28.567977 26078 hybrid_clock.cc:648] HybridClock initialized: now 1786395628567965 us; error 29 us; skew 500 ppm
I20260810 21:00:28.569178 26078 webserver.cc:460] Webserver started at http://127.14.200.2:45017/ using document root <none> and password file <none>
I20260810 21:00:28.569401 26078 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:28.569446 26078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:28.569620 26078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:28.570617 26078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "7fa6d0d64706414aba50cf61483870ae"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "fef23984dc3fc382e604c4df66f6c913"
server_key_iv: "25b24fa0496980a2b231af35c0937b10"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.570996 26078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "7fa6d0d64706414aba50cf61483870ae"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "fef23984dc3fc382e604c4df66f6c913"
server_key_iv: "25b24fa0496980a2b231af35c0937b10"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.572728 26078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:28.573457 26095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.573616 26078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:28.573714 26078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "7fa6d0d64706414aba50cf61483870ae"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "fef23984dc3fc382e604c4df66f6c913"
server_key_iv: "25b24fa0496980a2b231af35c0937b10"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.573802 26078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.576865 26078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.577135 26078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.577265 26078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.577471 26078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:28.577756 26078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:28.577809 26078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.577857 26078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:28.577883 26078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.583276 26078 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:33775
I20260810 21:00:28.583317 26217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:33775 every 8 connection(s)
I20260810 21:00:28.583608 26078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:00:28.584211 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26078
I20260810 21:00:28.584308 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:28.584510 15136 external_mini_cluster.cc:1428] Setting key d4d813aef615e9a8cc2eeef54cdce339
I20260810 21:00:28.585515 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--builtin_ntp_servers=127.14.200.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 21:00:28.586756 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.584232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:34025 (local address 127.14.200.62:34529)
0810 21:00:28.584303 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.584305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.584501 (+ 196us) server_negotiation.cc:407] Connection header received
0810 21:00:28.585092 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.585093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.585099 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.585169 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.586516 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.586518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.586518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.586522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.586536 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.586539 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.586579 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.586694 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.586706 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:00:28.587045 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.584179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34529 (local address 127.14.200.2:34025)
0810 21:00:28.584372 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:28.584414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:28.584981 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:28.585231 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:28.585236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:28.585244 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:28.586357 (+ 1113us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:28.586358 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:28.586393 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:28.586425 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:28.586597 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:28.586618 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:28.586621 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:00:28.586699 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 21:00:28.586757 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260810 21:00:28.587586 26218 heartbeater.cc:347] Connected to a master server at 127.14.200.62:34529
I20260810 21:00:28.587694 26218 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:28.587846 26218 heartbeater.cc:511] Master 127.14.200.62:34529 requested a full tablet report, sending...
I20260810 21:00:28.588207 25870 ts_manager.cc:194] Registered new tserver with Master: 7fa6d0d64706414aba50cf61483870ae (127.14.200.2:33775)
W20260810 21:00:28.664620 26221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:28.664836 26221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:28.664873 26221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:28.665967 26221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:28.666024 26221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:28.666066 26221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:28.667403 26221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:28.667659 26221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:28.667981 26221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:28.672844 26226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:28.672837 26229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.672853 26221 server_base.cc:1034] running on GCE node
W20260810 21:00:28.672837 26227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:28.673329 26221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:28.673614 26221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:28.674778 26221 hybrid_clock.cc:648] HybridClock initialized: now 1786395628674768 us; error 29 us; skew 500 ppm
I20260810 21:00:28.676107 26221 webserver.cc:460] Webserver started at http://127.14.200.3:37289/ using document root <none> and password file <none>
I20260810 21:00:28.676342 26221 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:28.676406 26221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:28.676540 26221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:28.677742 26221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "35ef8a3a2d0845cfbaccbd8b921b46fb"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "c5acdd3da3746a411dc0804dcb3a78b3"
server_key_iv: "7df257b1683eed71c6cc2f24988be927"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.678128 26221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "35ef8a3a2d0845cfbaccbd8b921b46fb"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "c5acdd3da3746a411dc0804dcb3a78b3"
server_key_iv: "7df257b1683eed71c6cc2f24988be927"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.679368 26221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:00:28.680078 26236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.680245 26221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:28.680346 26221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "35ef8a3a2d0845cfbaccbd8b921b46fb"
format_stamp: "Formatted at 2026-08-10 21:00:28 on dist-test-slave-1x4g"
server_key: "c5acdd3da3746a411dc0804dcb3a78b3"
server_key_iv: "7df257b1683eed71c6cc2f24988be927"
server_key_version: "encryptionkey@0"
I20260810 21:00:28.680440 26221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:28.683804 26221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:28.684074 26221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:28.684218 26221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:28.684427 26221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:28.684728 26221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:28.684780 26221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.684839 26221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:28.684866 26221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:28.690502 26221 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:35743
I20260810 21:00:28.690547 26361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:35743 every 8 connection(s)
I20260810 21:00:28.690843 26221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:28.691162 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26221
I20260810 21:00:28.691251 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:28.691444 15136 external_mini_cluster.cc:1428] Setting key ef86f717895e406b37eaaa67e1105299
I20260810 21:00:28.693771 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.691572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:43579 (local address 127.14.200.62:34529)
0810 21:00:28.691637 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.691639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.691945 (+ 306us) server_negotiation.cc:407] Connection header received
0810 21:00:28.692482 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.692482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.692488 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.692536 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.693593 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:28.693594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:28.693595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:00:28.693598 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:28.693613 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:00:28.693616 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:28.693665 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:28.693712 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 21:00:28.693723 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 21:00:28.693981 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:28.691540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34529 (local address 127.14.200.3:43579)
0810 21:00:28.691759 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:28.691800 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:28.692400 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:28.692583 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:28.692586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:28.692594 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:28.693456 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:00:28.693457 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:28.693483 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:00:28.693509 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:28.693667 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:28.693669 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:28.693671 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:00:28.693718 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 21:00:28.693762 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260810 21:00:28.694411 26362 heartbeater.cc:347] Connected to a master server at 127.14.200.62:34529
I20260810 21:00:28.694484 26362 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:28.694644 26362 heartbeater.cc:511] Master 127.14.200.62:34529 requested a full tablet report, sending...
I20260810 21:00:28.694983 25870 ts_manager.cc:194] Registered new tserver with Master: 35ef8a3a2d0845cfbaccbd8b921b46fb (127.14.200.3:35743)
I20260810 21:00:28.703182 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 21:00:28.704736 25915 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:00:28.704366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.14.200.62:34529)
0810 21:00:28.704412 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:28.704414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:28.704513 (+ 99us) server_negotiation.cc:407] Connection header received
0810 21:00:28.704558 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:28.704558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:28.704562 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:28.704607 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:28.704695 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54706: BlockingRecv error: recv got EOF from 127.0.0.1:54706 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 21:00:28.704885 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25938
I20260810 21:00:28.715199 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26078
I20260810 21:00:28.721953 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26221
I20260810 21:00:28.727568 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 25838
2026-08-10T21:00:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (532 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:28 dist-test-slave-1x4g krb5kdc[26372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:28 dist-test-slave-1x4g krb5kdc[26372](info): set up 2 sockets
Aug 10 21:00:28 dist-test-slave-1x4g krb5kdc[26372](info): commencing operation
krb5kdc: starting...
W20260810 21:00:29.761808 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:29 dist-test-slave-1x4g krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:00:29.888396 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45587
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:29.964376 26386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:29.964599 26386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:29.964636 26386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:29.965814 26386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:00:29.965873 26386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:29.965894 26386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:29.965914 26386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:00:29.965945 26386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:00:29.967356 26386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:29.967612 26386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:29.967842 26386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:29.972549 26395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:29.972638 26391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:29.972690 26386 server_base.cc:1034] running on GCE node
W20260810 21:00:29.972587 26392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:29.972995 26386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:29.973212 26386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:29.974354 26386 hybrid_clock.cc:648] HybridClock initialized: now 1786395629974346 us; error 28 us; skew 500 ppm
Aug 10 21:00:29 dist-test-slave-1x4g krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395629, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:29.976620 26386 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:00:29.977118 26386 webserver.cc:460] Webserver started at http://127.14.200.62:44127/ using document root <none> and password file <none>
I20260810 21:00:29.977349 26386 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:29.977416 26386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:29.977612 26386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:29.978615 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "d38becf67c164c9aabcea672e080cdd7"
format_stamp: "Formatted at 2026-08-10 21:00:29 on dist-test-slave-1x4g"
server_key: "5e5fabcbd94e0f2add62114f4d07f357"
server_key_iv: "1eacc81c9500b4f2f38c35ef7daf0eec"
server_key_version: "encryptionkey@0"
I20260810 21:00:29.978951 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "d38becf67c164c9aabcea672e080cdd7"
format_stamp: "Formatted at 2026-08-10 21:00:29 on dist-test-slave-1x4g"
server_key: "5e5fabcbd94e0f2add62114f4d07f357"
server_key_iv: "1eacc81c9500b4f2f38c35ef7daf0eec"
server_key_version: "encryptionkey@0"
I20260810 21:00:29.980195 26386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:00:29.980882 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:29.981052 26386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:29.981141 26386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "d38becf67c164c9aabcea672e080cdd7"
format_stamp: "Formatted at 2026-08-10 21:00:29 on dist-test-slave-1x4g"
server_key: "5e5fabcbd94e0f2add62114f4d07f357"
server_key_iv: "1eacc81c9500b4f2f38c35ef7daf0eec"
server_key_version: "encryptionkey@0"
I20260810 21:00:29.981191 26386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:30.009069 26386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:30.010761 26386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:30.010872 26386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:30.014789 26386 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45587
I20260810 21:00:30.014822 26459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45587 every 8 connection(s)
I20260810 21:00:30.015143 26386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:00:30.016031 26460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:30.018270 26460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: Bootstrap starting.
I20260810 21:00:30.018740 26460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:30.018965 26460 log.cc:824] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:30.019562 26460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: No bootstrap required, opened a new log
I20260810 21:00:30.020951 26460 raft_consensus.cc:348] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } }
I20260810 21:00:30.021076 26460 raft_consensus.cc:374] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:30.021097 26460 raft_consensus.cc:729] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38becf67c164c9aabcea672e080cdd7, State: Initialized, Role: FOLLOWER
I20260810 21:00:30.021219 26460 consensus_queue.cc:260] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } }
I20260810 21:00:30.021291 26460 raft_consensus.cc:388] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:00:30.021312 26460 raft_consensus.cc:482] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:00:30.021337 26460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:30.022004 26460 raft_consensus.cc:504] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } }
I20260810 21:00:30.022090 26460 leader_election.cc:302] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d38becf67c164c9aabcea672e080cdd7; no voters:
I20260810 21:00:30.022194 26460 leader_election.cc:288] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:00:30.022251 26463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:30.022423 26460 sys_catalog.cc:564] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:00:30.022440 26463 raft_consensus.cc:686] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [term 1 LEADER]: Becoming Leader. State: Replica: d38becf67c164c9aabcea672e080cdd7, State: Running, Role: LEADER
I20260810 21:00:30.022624 26463 consensus_queue.cc:237] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } }
I20260810 21:00:30.022956 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26386
I20260810 21:00:30.022961 26464 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d38becf67c164c9aabcea672e080cdd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } } }
I20260810 21:00:30.023031 26464 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:30.023066 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:00:30.023211 26465 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d38becf67c164c9aabcea672e080cdd7. Latest consensus state: current_term: 1 leader_uuid: "d38becf67c164c9aabcea672e080cdd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38becf67c164c9aabcea672e080cdd7" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45587 } } }
I20260810 21:00:30.023260 26465 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7 [sys.catalog]: This master's current role is: LEADER
I20260810 21:00:30.023347 15136 external_mini_cluster.cc:1428] Setting key 747581e1f3642500f7483b65672dd97d
I20260810 21:00:30.023586 26470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:00:30.024106 26470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:00:30.025590 26470 catalog_manager.cc:1347] Generated new cluster ID: b8a66e4ccc2f4aa694517c81bcde5d1f
I20260810 21:00:30.025637 26470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:30.034921 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.024003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.14.200.62:45587)
0810 21:00:30.024468 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.024476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.024504 (+ 28us) server_negotiation.cc:407] Connection header received
0810 21:00:30.024707 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.024717 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.024802 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.024962 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:30.025675 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.026998 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.028063 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.028631 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.028757 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.031285 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:30.031289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:30.031290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:30.031301 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:30.033289 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.033673 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.033676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.033676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.033791 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.033998 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.033998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.033999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.034298 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:30.034457 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.034554 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.034707 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":367,"thread_start_us":78,"threads_started":1}
I20260810 21:00:30.038749 26470 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:00:30.039170 26470 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:00:30.043159 26470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: Generated new TSK 0
I20260810 21:00:30.043303 26470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:00:30.075110 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:30.174391 26487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:30.174580 26487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:30.174603 26487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:30.175690 26487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:30.175729 26487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:30.175752 26487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:00:30.177284 26487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:30.177537 26487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:30.177783 26487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:30.182469 26493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:30.182538 26487 server_base.cc:1034] running on GCE node
W20260810 21:00:30.182457 26495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:30.182430 26492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:30.182847 26487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:30.183049 26487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:30.184178 26487 hybrid_clock.cc:648] HybridClock initialized: now 1786395630184169 us; error 25 us; skew 500 ppm
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:30.186745 26487 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:00:30.187151 26487 webserver.cc:460] Webserver started at http://127.14.200.1:43259/ using document root <none> and password file <none>
I20260810 21:00:30.187386 26487 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:30.187461 26487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:30.187608 26487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:30.188580 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "5147591dd1d54c108533053e5141ad19"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "ca37964796f4975c9be5f9967a10f655"
server_key_iv: "02c4319015edf996ace50ab80bbfedc4"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.188951 26487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "5147591dd1d54c108533053e5141ad19"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "ca37964796f4975c9be5f9967a10f655"
server_key_iv: "02c4319015edf996ace50ab80bbfedc4"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.190146 26487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:00:30.190798 26504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.190953 26487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.191045 26487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "5147591dd1d54c108533053e5141ad19"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "ca37964796f4975c9be5f9967a10f655"
server_key_iv: "02c4319015edf996ace50ab80bbfedc4"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.191125 26487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:30.222113 26487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:30.223961 26487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:30.224148 26487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:30.224387 26487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:30.224696 26487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:30.224746 26487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.224777 26487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:30.224823 26487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.230384 26487 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:35495
I20260810 21:00:30.230450 26624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:35495 every 8 connection(s)
I20260810 21:00:30.230758 26487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:30.239074 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.231424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33009 (local address 127.14.200.62:45587)
0810 21:00:30.231491 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.231493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.231700 (+ 207us) server_negotiation.cc:407] Connection header received
0810 21:00:30.232159 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.232160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.232168 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.232220 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:30.233037 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.233522 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.234437 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.234623 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.234671 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.236665 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:30.236666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:30.236666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:30.236676 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:30.237933 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.238286 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.238287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.238287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.238337 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.238554 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.238555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.238555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.238653 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:30.238705 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.238952 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.239015 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:00:30.239224 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.231417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45587 (local address 127.14.200.1:33009)
0810 21:00:30.231581 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.231621 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.232081 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.232267 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.232270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.232769 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:30.232959 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.232964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.233601 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.233601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.234363 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.234365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.234694 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.234695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.234793 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.235289 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:30.235297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:30.236573 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:30.238004 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.238011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.238013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.238222 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.238395 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.238396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.238396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.238491 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.238709 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:30.238714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:30.238833 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.238919 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.239026 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260810 21:00:30.239750 26625 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45587
I20260810 21:00:30.239826 26625 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:30.240008 26625 heartbeater.cc:511] Master 127.14.200.62:45587 requested a full tablet report, sending...
I20260810 21:00:30.240549 26421 ts_manager.cc:194] Registered new tserver with Master: 5147591dd1d54c108533053e5141ad19 (127.14.200.1:35495)
I20260810 21:00:30.240748 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26487
I20260810 21:00:30.240876 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:00:30.241073 15136 external_mini_cluster.cc:1428] Setting key e01dbc6dbcdebd76b1cfd3bc503adc7f
I20260810 21:00:30.241380 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:33009
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:00:30.267158 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:30.346614 26631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:30.346814 26631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:30.346836 26631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:30.348008 26631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:30.348058 26631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:30.348083 26631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:00:30.349408 26631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:30.349632 26631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:30.349836 26631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:30.354296 26636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:30.354328 26637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:30.354306 26640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:30.354601 26631 server_base.cc:1034] running on GCE node
I20260810 21:00:30.354802 26631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:30.355038 26631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:30.356226 26631 hybrid_clock.cc:648] HybridClock initialized: now 1786395630356210 us; error 28 us; skew 500 ppm
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:30.358711 26631 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:00:30.359109 26631 webserver.cc:460] Webserver started at http://127.14.200.2:46783/ using document root <none> and password file <none>
I20260810 21:00:30.359325 26631 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:30.359390 26631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:30.359539 26631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:30.360477 26631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "68285efadf7b42b6a49c9ebd2620ba5b"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "0101b3275dbf93b9e20ae3b042c72917"
server_key_iv: "c8289ba1a309a5db9ed0f7ab0dd96ec0"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.360913 26631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "68285efadf7b42b6a49c9ebd2620ba5b"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "0101b3275dbf93b9e20ae3b042c72917"
server_key_iv: "c8289ba1a309a5db9ed0f7ab0dd96ec0"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.362202 26631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:00:30.362862 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.363024 26631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:00:30.363116 26631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "68285efadf7b42b6a49c9ebd2620ba5b"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "0101b3275dbf93b9e20ae3b042c72917"
server_key_iv: "c8289ba1a309a5db9ed0f7ab0dd96ec0"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.363193 26631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:30.372946 26631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:30.374627 26631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:30.374774 26631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:30.374971 26631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:30.375258 26631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:30.375319 26631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.375355 26631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:30.375393 26631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.381126 26631 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:38907
I20260810 21:00:30.381158 26770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:38907 every 8 connection(s)
I20260810 21:00:30.381507 26631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:30.390350 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.382221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60925 (local address 127.14.200.62:45587)
0810 21:00:30.382282 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.382286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.382474 (+ 188us) server_negotiation.cc:407] Connection header received
0810 21:00:30.382992 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.382994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.383008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.383061 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:30.383967 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.384471 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.385480 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.385671 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.385770 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.387782 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:30.387783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:30.387784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:30.387790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:30.389193 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.389563 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.389564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.389564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.389616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.389829 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.389829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.389830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.389947 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:30.389998 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.390229 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.390291 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 21:00:30.390470 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.382203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45587 (local address 127.14.200.2:60925)
0810 21:00:30.382367 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.382399 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.382869 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.383125 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.383134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.383679 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:30.383851 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.383855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.384565 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.384566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.385382 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.385384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.385783 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.385784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.385899 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.386372 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:30.386380 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:30.387699 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:30.389279 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.389285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.389287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.389489 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.389671 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.389672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.389672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.389759 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.390000 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:30.390004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:30.390112 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.390189 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.390307 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260810 21:00:30.390985 26771 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45587
I20260810 21:00:30.391078 26771 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:30.391263 26771 heartbeater.cc:511] Master 127.14.200.62:45587 requested a full tablet report, sending...
I20260810 21:00:30.391597 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26631
I20260810 21:00:30.391664 26420 ts_manager.cc:194] Registered new tserver with Master: 68285efadf7b42b6a49c9ebd2620ba5b (127.14.200.2:38907)
I20260810 21:00:30.391695 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:00:30.391897 15136 external_mini_cluster.cc:1428] Setting key 2b2b990d7795b993c820c99a68ed033d
I20260810 21:00:30.392105 26420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:60925
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:00:30.417150 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:00:30.495230 26776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:00:30.495428 26776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:00:30.495458 26776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:00:30.496490 26776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:00:30.496521 26776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:00:30.496546 26776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:00:30.498000 26776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:00:30.498189 26776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:00:30.498363 26776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:00:30.503626 26784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:30.503664 26781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:00:30.503626 26782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:00:30.503890 26776 server_base.cc:1034] running on GCE node
I20260810 21:00:30.504104 26776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:00:30.504338 26776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:00:30.505472 26776 hybrid_clock.cc:648] HybridClock initialized: now 1786395630505460 us; error 27 us; skew 500 ppm
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:00:30.507963 26776 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:00:30.508615 26776 webserver.cc:460] Webserver started at http://127.14.200.3:34187/ using document root <none> and password file <none>
I20260810 21:00:30.508903 26776 fs_manager.cc:359] Metadata directory not provided
I20260810 21:00:30.508960 26776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:00:30.509104 26776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:00:30.510146 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "fd9bd708678a4b829629a1d275703047"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "439122103d767b33b9faf6b4ae20a647"
server_key_iv: "6de0fa94442955625f7a051e67014396"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.510509 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "fd9bd708678a4b829629a1d275703047"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "439122103d767b33b9faf6b4ae20a647"
server_key_iv: "6de0fa94442955625f7a051e67014396"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.511893 26776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:00:30.512573 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.512737 26776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:00:30.512884 26776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "fd9bd708678a4b829629a1d275703047"
format_stamp: "Formatted at 2026-08-10 21:00:30 on dist-test-slave-1x4g"
server_key: "439122103d767b33b9faf6b4ae20a647"
server_key_iv: "6de0fa94442955625f7a051e67014396"
server_key_version: "encryptionkey@0"
I20260810 21:00:30.512959 26776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:00:30.533156 26776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:00:30.534807 26776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:00:30.534965 26776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:00:30.535187 26776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:00:30.535494 26776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:00:30.535548 26776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.535588 26776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:00:30.535614 26776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:00:30.541275 26776 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:37883
I20260810 21:00:30.541311 26914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:37883 every 8 connection(s)
I20260810 21:00:30.541640 26776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:00:30.541767 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 26776
I20260810 21:00:30.541869 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:00:30.542099 15136 external_mini_cluster.cc:1428] Setting key 69bb083a175c511993d0dc9e840a8c6d
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395630, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:00:30.550981 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.542647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38493 (local address 127.14.200.62:45587)
0810 21:00:30.542693 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.542695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.542704 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:30.543471 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.543472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.543484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.543565 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:30.544494 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.545071 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.546033 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.546214 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.546290 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.548417 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:30.548419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:30.548420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:30.548427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:30.549620 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.550028 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.550029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.550029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.550086 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.550344 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.550345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.550346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.550460 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:30.550528 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.550800 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.550877 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:00:30.551131 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.542345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45587 (local address 127.14.200.3:38493)
0810 21:00:30.542548 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.542590 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.543357 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.543637 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.543641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.544207 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:00:30.544386 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.544390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.545161 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.545162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.545920 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.545922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.546304 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.546305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.546419 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.546988 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:00:30.546996 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:00:30.548323 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:00:30.549703 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.549710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.549712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.549930 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.550152 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:00:30.550153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:00:30.550153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:00:30.550271 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:00:30.550532 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:00:30.550539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:00:30.550666 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.550763 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.550922 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":126,"threads_started":1}
I20260810 21:00:30.551568 26915 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45587
I20260810 21:00:30.551666 26915 heartbeater.cc:464] Registering TS with master...
I20260810 21:00:30.551831 26915 heartbeater.cc:511] Master 127.14.200.62:45587 requested a full tablet report, sending...
I20260810 21:00:30.552218 26421 ts_manager.cc:194] Registered new tserver with Master: fd9bd708678a4b829629a1d275703047 (127.14.200.3:37883)
I20260810 21:00:30.552579 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:38493
I20260810 21:00:30.553783 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:00:30.560326 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.554982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.14.200.62:45587)
0810 21:00:30.555045 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.555047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.555141 (+ 94us) server_negotiation.cc:407] Connection header received
0810 21:00:30.555161 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.555161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.555169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.555213 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:00:30.555701 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.556136 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.556893 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.557068 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.557114 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.558217 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:00:30.558218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:00:30.558218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:00:30.558222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:00:30.559299 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.559611 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.559612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.559612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.559659 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:00:30.559816 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:00:30.559817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:00:30.559817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:00:30.559937 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:00:30.560007 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.560175 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.560230 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:00:30.561666 26421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:00:30.563192 26421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:00:30.569618 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:38907 (local address 127.0.0.1:39152)
0810 21:00:30.565841 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.565849 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.565897 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.566102 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.566103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.566106 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:30.566173 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.566176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567402 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.567403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.568707 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.568711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.569376 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.569377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.569423 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.569443 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.569475 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.569528 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":58,"thread_start_us":80,"threads_started":1}
I20260810 21:00:30.569641 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.14.200.3:37883)
0810 21:00:30.565621 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.565625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.565767 (+ 142us) server_negotiation.cc:407] Connection header received
0810 21:00:30.565838 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.565840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.565858 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.565931 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:30.566243 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567384 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.568607 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.569250 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.569320 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.569376 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.569532 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.569569 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260810 21:00:30.569618 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:37883 (local address 127.0.0.1:34068)
0810 21:00:30.565720 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.565728 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.565768 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.566095 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.566098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.566100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:30.566163 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.566165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567493 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.567494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.568526 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.568529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.569340 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.569342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.569418 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.569431 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.569500 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.569529 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":78,"thread_start_us":86,"threads_started":1}
I20260810 21:00:30.570040 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39152 (local address 127.14.200.2:38907)
0810 21:00:30.565792 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.565796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.565908 (+ 112us) server_negotiation.cc:407] Connection header received
0810 21:00:30.565958 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.565960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.565978 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.566088 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:30.566366 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567298 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.568831 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.569271 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.569839 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.569907 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.569930 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.569969 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260810 21:00:30.570385 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:35495 (local address 127.0.0.1:44076)
0810 21:00:30.565527 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.565565 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.565631 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.566195 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.566196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.566198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:30.566239 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.566240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567475 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.567475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.568721 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.568723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.570126 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.570127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.570194 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.570198 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.570263 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.570321 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":37,"thread_start_us":47,"threads_started":1}
I20260810 21:00:30.570403 26842 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 (DEFAULT_TABLE table=test-table [id=9673c5bc0598416e87dd8a6b1ee5d0b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:30.570664 26842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0fb5e35d034688823b586d4dc35e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:30.570681 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.565851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.14.200.1:35495)
0810 21:00:30.566052 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.566056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.566065 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:00:30.566102 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.566104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.566119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.566199 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:30.566287 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.567372 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.568821 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.569626 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.570477 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.570540 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.570566 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.570606 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260810 21:00:30.570698 26698 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 (DEFAULT_TABLE table=test-table [id=9673c5bc0598416e87dd8a6b1ee5d0b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:30.570972 26698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0fb5e35d034688823b586d4dc35e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:30.573436 26936 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Bootstrap starting.
I20260810 21:00:30.573800 26553 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 (DEFAULT_TABLE table=test-table [id=9673c5bc0598416e87dd8a6b1ee5d0b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:00:30.574043 26936 tablet_bootstrap.cc:654] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:30.574090 26553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0fb5e35d034688823b586d4dc35e48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:00:30.574536 26936 log.cc:824] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:30.575227 26936 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: No bootstrap required, opened a new log
I20260810 21:00:30.575302 26936 ts_tablet_manager.cc:1397] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:00:30.576548 26936 raft_consensus.cc:348] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.576726 26936 raft_consensus.cc:374] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:30.576727 26938 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Bootstrap starting.
I20260810 21:00:30.576790 26936 raft_consensus.cc:729] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68285efadf7b42b6a49c9ebd2620ba5b, State: Initialized, Role: FOLLOWER
I20260810 21:00:30.576926 26936 consensus_queue.cc:260] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.577203 26936 ts_tablet_manager.cc:1428] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:00:30.577201 26935 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Bootstrap starting.
I20260810 21:00:30.577342 26771 heartbeater.cc:503] Master 127.14.200.62:45587 was elected leader, sending a full tablet report...
I20260810 21:00:30.577414 26938 tablet_bootstrap.cc:654] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:30.577904 26935 tablet_bootstrap.cc:654] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Neither blocks nor log segments found. Creating new log.
I20260810 21:00:30.578007 26938 log.cc:824] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:30.578398 26935 log.cc:824] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Log is configured to *not* fsync() on all Append() calls
I20260810 21:00:30.578738 26938 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: No bootstrap required, opened a new log
I20260810 21:00:30.578819 26938 ts_tablet_manager.cc:1397] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:00:30.579136 26935 tablet_bootstrap.cc:492] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: No bootstrap required, opened a new log
I20260810 21:00:30.579200 26935 ts_tablet_manager.cc:1397] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:00:30.580643 26938 raft_consensus.cc:348] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.580830 26938 raft_consensus.cc:374] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:30.580878 26938 raft_consensus.cc:729] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5147591dd1d54c108533053e5141ad19, State: Initialized, Role: FOLLOWER
I20260810 21:00:30.580986 26938 consensus_queue.cc:260] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.581061 26935 raft_consensus.cc:348] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.581228 26935 raft_consensus.cc:374] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:00:30.581276 26935 raft_consensus.cc:729] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9bd708678a4b829629a1d275703047, State: Initialized, Role: FOLLOWER
I20260810 21:00:30.581295 26625 heartbeater.cc:503] Master 127.14.200.62:45587 was elected leader, sending a full tablet report...
I20260810 21:00:30.581385 26935 consensus_queue.cc:260] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.581516 26938 ts_tablet_manager.cc:1428] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 21:00:30.581650 26935 ts_tablet_manager.cc:1428] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 21:00:30.581777 26915 heartbeater.cc:503] Master 127.14.200.62:45587 was elected leader, sending a full tablet report...
W20260810 21:00:30.632146 26772 tablet.cc:2369] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:00:30.731544 26627 tablet.cc:2369] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:00:30.792322 26916 tablet.cc:2369] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:00:30.793112 26944 raft_consensus.cc:482] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:00:30.793200 26944 raft_consensus.cc:504] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.793604 26944 leader_election.cc:288] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5147591dd1d54c108533053e5141ad19 (127.14.200.1:35495), 68285efadf7b42b6a49c9ebd2620ba5b (127.14.200.2:38907)
I20260810 21:00:30.796372 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.793759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:35495 (local address 127.14.200.3:51867)
0810 21:00:30.793819 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.793831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.793880 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.794076 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.794077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.794079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:30.794155 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.794157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.794859 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.794860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.795581 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.795583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.796107 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.796109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.796195 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.796199 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.796253 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.796286 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 21:00:30.796386 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.793879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:51867 (local address 127.14.200.1:35495)
0810 21:00:30.793927 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.793929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.793937 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:00:30.793968 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.793969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.793979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.794046 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:30.794226 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.794797 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.795671 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.796035 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.796088 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.796096 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.796269 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.796318 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:00:30.796489 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.793858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:58435 (local address 127.14.200.2:38907)
0810 21:00:30.793902 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.793904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.794071 (+ 167us) server_negotiation.cc:407] Connection header received
0810 21:00:30.794107 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.794108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.794119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.794177 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:00:30.794338 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.794889 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.795733 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.796115 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.796183 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.796191 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.796357 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.796402 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:00:30.796519 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.793815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:38907 (local address 127.14.200.3:58435)
0810 21:00:30.793986 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.793996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:00:30.794062 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:00:30.794199 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:00:30.794200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:00:30.794202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:00:30.794272 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.794274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.794962 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.794962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.795643 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:00:30.795645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.796196 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.796197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:00:30.796279 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.796283 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:00:30.796338 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 21:00:30.796375 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260810 21:00:30.796909 26576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0fb5e35d034688823b586d4dc35e48" candidate_uuid: "fd9bd708678a4b829629a1d275703047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5147591dd1d54c108533053e5141ad19" is_pre_election: true
I20260810 21:00:30.796921 26722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0fb5e35d034688823b586d4dc35e48" candidate_uuid: "fd9bd708678a4b829629a1d275703047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" is_pre_election: true
I20260810 21:00:30.797072 26576 raft_consensus.cc:2454] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9bd708678a4b829629a1d275703047 in term 0.
I20260810 21:00:30.797072 26722 raft_consensus.cc:2454] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9bd708678a4b829629a1d275703047 in term 0.
I20260810 21:00:30.797300 26793 leader_election.cc:302] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68285efadf7b42b6a49c9ebd2620ba5b, fd9bd708678a4b829629a1d275703047; no voters:
I20260810 21:00:30.797446 26944 raft_consensus.cc:2781] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:00:30.797516 26944 raft_consensus.cc:482] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:00:30.797560 26944 raft_consensus.cc:3037] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:30.798290 26944 raft_consensus.cc:504] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.798418 26944 leader_election.cc:288] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [CANDIDATE]: Term 1 election: Requested vote from peers 5147591dd1d54c108533053e5141ad19 (127.14.200.1:35495), 68285efadf7b42b6a49c9ebd2620ba5b (127.14.200.2:38907)
I20260810 21:00:30.798555 26722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0fb5e35d034688823b586d4dc35e48" candidate_uuid: "fd9bd708678a4b829629a1d275703047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68285efadf7b42b6a49c9ebd2620ba5b"
I20260810 21:00:30.798574 26576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0fb5e35d034688823b586d4dc35e48" candidate_uuid: "fd9bd708678a4b829629a1d275703047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5147591dd1d54c108533053e5141ad19"
I20260810 21:00:30.798625 26722 raft_consensus.cc:3037] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:30.798647 26576 raft_consensus.cc:3037] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:00:30.799353 26576 raft_consensus.cc:2454] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9bd708678a4b829629a1d275703047 in term 1.
I20260810 21:00:30.799365 26722 raft_consensus.cc:2454] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9bd708678a4b829629a1d275703047 in term 1.
I20260810 21:00:30.799537 26794 leader_election.cc:302] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5147591dd1d54c108533053e5141ad19, fd9bd708678a4b829629a1d275703047; no voters:
I20260810 21:00:30.799638 26944 raft_consensus.cc:2781] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:00:30.799798 26944 raft_consensus.cc:686] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 1 LEADER]: Becoming Leader. State: Replica: fd9bd708678a4b829629a1d275703047, State: Running, Role: LEADER
I20260810 21:00:30.799911 26944 consensus_queue.cc:237] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } }
I20260810 21:00:30.800716 26420 catalog_manager.cc:5495] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd9bd708678a4b829629a1d275703047 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "fd9bd708678a4b829629a1d275703047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd9bd708678a4b829629a1d275703047" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 37883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 } health_report { overall_health: UNKNOWN } } }
I20260810 21:00:30.857496 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.855195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.14.200.3:37883)
0810 21:00:30.855293 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 21:00:30.855295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:00:30.855302 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:00:30.855382 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:00:30.855383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:00:30.855413 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:00:30.855474 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:00:30.855605 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.856054 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.856758 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:00:30.856981 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:00:30.857036 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:00:30.857173 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:00:30.857269 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:00:30.857324 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:00:30.857353 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 21:00:30.857367 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 21:00:30.859316 26576 raft_consensus.cc:1261] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 1 FOLLOWER]: Refusing update from remote peer fd9bd708678a4b829629a1d275703047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:30.859344 26722 raft_consensus.cc:1261] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 1 FOLLOWER]: Refusing update from remote peer fd9bd708678a4b829629a1d275703047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:00:30.859727 26944 consensus_queue.cc:1035] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5147591dd1d54c108533053e5141ad19" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 35495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:30.859894 26946 consensus_queue.cc:1035] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68285efadf7b42b6a49c9ebd2620ba5b" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 38907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:00:30.862135 26950 mvcc.cc:205] Tried to move back new op lower bound from 7317076503997038592 to 7317076503756996608. Current Snapshot: MvccSnapshot[applied={T|T < 7317076503997038592}]
I20260810 21:00:30.862478 26951 mvcc.cc:205] Tried to move back new op lower bound from 7317076503997038592 to 7317076503756996608. Current Snapshot: MvccSnapshot[applied={T|T < 7317076503997038592}]
I20260810 21:00:30.863454 26952 mvcc.cc:205] Tried to move back new op lower bound from 7317076503997038592 to 7317076503756996608. Current Snapshot: MvccSnapshot[applied={T|T < 7317076503997038592 or (T in {7317076503997038592})}]
I20260810 21:00:30.864729 26420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:30.864851 26420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:00:30.865584 26420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d38becf67c164c9aabcea672e080cdd7: Sending DeleteTablet for 3 replicas of tablet 8d0fb5e35d034688823b586d4dc35e48
I20260810 21:00:30.865896 26553 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:44076
I20260810 21:00:30.865924 26698 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:39152
I20260810 21:00:30.866146 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26487
I20260810 21:00:30.866226 26960 tablet_replica.cc:330] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19: stopping tablet replica
I20260810 21:00:30.866217 26842 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0fb5e35d034688823b586d4dc35e48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:00:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:34068
I20260810 21:00:30.866392 26960 raft_consensus.cc:2229] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:30.866430 26961 tablet_replica.cc:330] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: stopping tablet replica
I20260810 21:00:30.866568 26960 raft_consensus.cc:2258] T 8d0fb5e35d034688823b586d4dc35e48 P 5147591dd1d54c108533053e5141ad19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:30.867412 26961 raft_consensus.cc:2229] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:00:30.867527 26961 raft_consensus.cc:2258] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:30.867982 26961 ts_tablet_manager.cc:1905] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:00:30.870242 26961 ts_tablet_manager.cc:1918] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:30.870317 26961 log.cc:1196] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/8d0fb5e35d034688823b586d4dc35e48
I20260810 21:00:30.870544 26961 ts_tablet_manager.cc:1939] T 8d0fb5e35d034688823b586d4dc35e48 P 68285efadf7b42b6a49c9ebd2620ba5b: Deleting consensus metadata
I20260810 21:00:30.870909 26404 catalog_manager.cc:4841] TS 68285efadf7b42b6a49c9ebd2620ba5b (127.14.200.2:38907): tablet 8d0fb5e35d034688823b586d4dc35e48 (table test-table [id=9673c5bc0598416e87dd8a6b1ee5d0b5]) successfully deleted
I20260810 21:00:30.872982 26962 tablet_replica.cc:330] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: stopping tablet replica
I20260810 21:00:30.873085 26962 raft_consensus.cc:2229] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:00:30.873170 26962 raft_consensus.cc:2258] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:00:30.873598 26962 ts_tablet_manager.cc:1905] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 21:00:30.874222 26405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:35495 (error 108)
I20260810 21:00:30.874684 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:00:30.874563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:35495 (local address 127.0.0.1:44090)
0810 21:00:30.874614 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 21:00:30.874648 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:35495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 21:00:30.874781 26405 catalog_manager.cc:4584] TS 5147591dd1d54c108533053e5141ad19 (127.14.200.1:35495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d0fb5e35d034688823b586d4dc35e48: Network error: Client connection negotiation failed: client connection to 127.14.200.1:35495: connect: Connection refused (error 111)
I20260810 21:00:30.875080 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26631
I20260810 21:00:30.875150 26962 ts_tablet_manager.cc:1918] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:00:30.877074 26962 log.cc:1196] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/8d0fb5e35d034688823b586d4dc35e48
I20260810 21:00:30.877355 26962 ts_tablet_manager.cc:1939] T 8d0fb5e35d034688823b586d4dc35e48 P fd9bd708678a4b829629a1d275703047: Deleting consensus metadata
I20260810 21:00:30.877708 26406 catalog_manager.cc:4841] TS fd9bd708678a4b829629a1d275703047 (127.14.200.3:37883): tablet 8d0fb5e35d034688823b586d4dc35e48 (table test-table [id=9673c5bc0598416e87dd8a6b1ee5d0b5]) successfully deleted
I20260810 21:00:30.887233 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26776
I20260810 21:00:30.893711 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 26386
2026-08-10T21:00:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2167 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26966](info): set up 2 sockets
Aug 10 21:00:30 dist-test-slave-1x4g krb5kdc[26966](info): commencing operation
krb5kdc: starting...
W20260810 21:00:31.931283 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:00:32 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:00:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:00:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangeradmin_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangeradmin_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangerlookup_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangerlookup_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
I20260810 21:00:32.088382 15136 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrRlFwH/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/postgres -l logfile start
2026-08-10 21:00:34.438 UTC [26998] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 21:00:34.438 UTC [26998] LOG: listening on IPv4 address "127.14.200.20", port 57553
2026-08-10 21:00:34.442 UTC [26998] LOG: listening on Unix socket "/tmp/.s.PGSQL.57553"
2026-08-10 21:00:34.463 UTC [27001] LOG: database system was shut down at 2026-08-10 21:00:33 UTC
2026-08-10 21:00:34.467 UTC [26998] LOG: database system is ready to accept connections
I20260810 21:00:35.426419 15136 mini_postgres.cc:96] Postgres bound to 57553
2026-08-10 21:00:35.434 UTC [27009] FATAL: database "slave" does not exist
127.14.200.20:57553 - accepting connections
I20260810 21:00:35.435611 15136 mini_ranger.cc:161] Starting Ranger...
I20260810 21:00:35.450691 15136 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 21:00:35.907435 15136 mini_ranger.cc:87] Created ranger Postgres database
I20260810 21:00:35.907522 15136 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin
2026-08-10 21:00:36,280 [I] DB FLAVOR :POSTGRES
2026-08-10 21:00:36,281 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 21:00:36,281 [I] Checking connection..
2026-08-10 21:00:36,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 21:00:36,569 [I] Checking connection passed.
2026-08-10 21:00:36,570 [I] --------- Verifying version history table ---------
2026-08-10 21:00:36,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:00:36,806 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 21:00:36,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:00:37,045 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 21:00:37,045 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 21:00:37,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 21:00:37.278 UTC [27079] WARNING: there is no transaction in progress
2026-08-10 21:00:37,285 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 21:00:37,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:00:37,504 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 21:00:37,504 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 21:00:37,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 21:00:37,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 21:00:37,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-10 21:00:38,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 21:00:38,458 [I] Table x_portal_user does not exist in database ranger
2026-08-10 21:00:38,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 21:00:38,676 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 21:00:38,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 21:00:38,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 21:00:39,161 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 21:00:39,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 21:00:40.091 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.106 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.123 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.139 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.155 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.175 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.229 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.236 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.250 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.260 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.270 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.284 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.294 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.300 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.307 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40.314 UTC [27197] WARNING: there is no transaction in progress
2026-08-10 21:00:40,925 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 21:00:40,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 21:00:41,143 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 21:00:41,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 21:00:41,373 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 21:00:41,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 21:00:41,615 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 21:00:41,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 21:00:41,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 21:00:42,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-10 21:00:42,311 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 21:00:42,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 21:00:42,544 [I] DB_PATCHES have already been applied
I20260810 21:00:42.550046 15136 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrRlFwH/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 21:00:42.550134 15136 mini_ranger.cc:193] Using host: 127.14.200.20
I20260810 21:00:42.554092 15136 mini_ranger.cc:227] Ranger admin URL: http://127.14.200.20:47229
Aug 10, 2026 9:00:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 9:00:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 9:00:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 9:00:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 9:00:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 9:00:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 9:00:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 21:00:43.712662 15136 mini_ranger.cc:160] Time spent starting Ranger: real 8.277s user 0.001s sys 0.010s
Aug 10, 2026 9:01:04 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 9:01:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 9:01:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 9:01:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 9:01:05 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 21:01:05.705369 15136 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangerkms_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/rangerkms_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.200.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.200.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 21:01:05 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395665, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 21:01:05.800010 15136 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 21:01:05.810679 15136 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 21:01:06.615482 15136 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 21:01:06.615571 15136 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms
I20260810 21:01:06.615594 15136 mini_ranger_kms.cc:227] Using postgres at 127.14.200.20:57553
2026-08-10 21:01:07,229 [I] DB FLAVOR :POSTGRES
2026-08-10 21:01:07,229 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 21:01:07,229 [I] Checking connection
2026-08-10 21:01:07,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 21:01:07,509 [I] connection success
2026-08-10 21:01:07,509 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 21:01:07,509 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 21:01:07,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 21:01:07,761 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 21:01:07,761 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 21:01:07,762 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 21:01:07,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:57553/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 21:01:08,016 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 21:01:08.482372 15136 mini_ranger_kms.cc:310] Created kms service
I20260810 21:01:08.690845 15136 mini_ranger_kms.cc:326] Created kudu user
I20260810 21:01:08.731772 15136 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 21:01:08 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395665, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:09.130897 15136 mini_ranger_kms.cc:384] Added ranger policy
I20260810 21:01:09.131038 15136 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 21:01:09.131060 15136 mini_ranger_kms.cc:240] Using host: 127.14.200.20
I20260810 21:01:09.135095 15136 mini_ranger_kms.cc:276] Ranger KMS PID: 27387
I20260810 21:01:09.135226 15136 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.200.20:35565
Aug 10, 2026 9:01:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 9:01:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 9:01:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 9:01:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 9:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 9:01:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 21:01:11.434798 15136 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.635s user 0.000s sys 0.465s
I20260810 21:01:11.434917 15136 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 21:01:11.435014 15136 mini_ranger_kms.cc:401] 127.14.200.20:35565/kms/v1/keys
Aug 10 21:01:11 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786395671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 21:01:12 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
Aug 10 21:01:12 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395671, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
Aug 10 21:01:15 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:01:15.109508 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:37387
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:37387
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:35565/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:01:15.216907 27443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:01:15.217105 27443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:01:15.217127 27443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:01:15.218315 27443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:01:15.218359 27443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:01:15.218374 27443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:01:15.218385 27443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:01:15.218430 27443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:01:15.220453 27443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.200.20:35565/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:37387
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:37387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:01:15.220727 27443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:01:15.220988 27443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:01:15.226516 27452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:15.226547 27449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:15.226722 27448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:15.226622 27443 server_base.cc:1034] running on GCE node
I20260810 21:01:15.226941 27443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:01:15.227229 27443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:01:15.228407 27443 hybrid_clock.cc:648] HybridClock initialized: now 1786395675228376 us; error 48 us; skew 500 ppm
Aug 10 21:01:15 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395675, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:15.242527 27443 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:01:15.243180 27443 webserver.cc:460] Webserver started at http://127.14.200.62:35263/ using document root <none> and password file <none>
I20260810 21:01:15.243412 27443 fs_manager.cc:359] Metadata directory not provided
I20260810 21:01:15.243449 27443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:01:15.243583 27443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:01:15 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395675, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:15.304030 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "386ba12521e449c5acaccadee93e3725"
format_stamp: "Formatted at 2026-08-10 21:01:15 on dist-test-slave-1x4g"
server_key: "d420db7f69a601d87aba34e1fdbc917d"
server_key_iv: "98436cb61abe42da881c11564abba3c8"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:15.304489 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "386ba12521e449c5acaccadee93e3725"
format_stamp: "Formatted at 2026-08-10 21:01:15 on dist-test-slave-1x4g"
server_key: "d420db7f69a601d87aba34e1fdbc917d"
server_key_iv: "98436cb61abe42da881c11564abba3c8"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:15.306403 27443 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 21:01:15.321316 27459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:15.321583 27443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:01:15.321713 27443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "386ba12521e449c5acaccadee93e3725"
format_stamp: "Formatted at 2026-08-10 21:01:15 on dist-test-slave-1x4g"
server_key: "d420db7f69a601d87aba34e1fdbc917d"
server_key_iv: "98436cb61abe42da881c11564abba3c8"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:15.321839 27443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:01:15.339080 27443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:01:15.341431 27443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:01:15.341640 27443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:01:15.352802 27443 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:37387
I20260810 21:01:15.352917 27515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:37387 every 8 connection(s)
I20260810 21:01:15.353317 27443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:01:15.354450 27516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:01:15.356295 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 27443
I20260810 21:01:15.356410 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:01:15.356628 15136 external_mini_cluster.cc:1428] Setting key fe0af155438c2bf250901ecbd796bb57
I20260810 21:01:15.358398 27516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: Bootstrap starting.
I20260810 21:01:15.359608 27516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: Neither blocks nor log segments found. Creating new log.
I20260810 21:01:15.360296 27516 log.cc:824] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: Log is configured to *not* fsync() on all Append() calls
I20260810 21:01:15.361212 27516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: No bootstrap required, opened a new log
Aug 10 21:01:15 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:01:15.364405 27516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } }
I20260810 21:01:15.364573 27516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:01:15.364604 27516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386ba12521e449c5acaccadee93e3725, State: Initialized, Role: FOLLOWER
I20260810 21:01:15.364792 27516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } }
I20260810 21:01:15.364913 27516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:01:15.364959 27516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:01:15.365000 27516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:01:15.365773 27516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } }
I20260810 21:01:15.365876 27516 leader_election.cc:302] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386ba12521e449c5acaccadee93e3725; no voters:
I20260810 21:01:15.366098 27516 leader_election.cc:288] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:01:15.366340 27516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:01:15.366355 27522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:01:15.367169 27522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [term 1 LEADER]: Becoming Leader. State: Replica: 386ba12521e449c5acaccadee93e3725, State: Running, Role: LEADER
I20260810 21:01:15.367509 27522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } }
I20260810 21:01:15.368470 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:15.357278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.14.200.62:37387)
0810 21:01:15.357639 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:15.357648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:15.357705 (+ 57us) server_negotiation.cc:407] Connection header received
0810 21:01:15.358297 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:15.358304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:15.358353 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:15.358509 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:15.359167 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:15.360408 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:15.361374 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:15.361874 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:15.362029 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:15.364298 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:15.364302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:15.364302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:15.364314 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:15.366705 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:15.367075 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:15.367078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:15.367078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:15.367229 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:15.367456 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:15.367457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:15.367457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:15.367741 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:15.367887 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:15.368007 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 21:01:15.368116 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":125,"threads_started":1}
I20260810 21:01:15.368655 27521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386ba12521e449c5acaccadee93e3725. Latest consensus state: current_term: 1 leader_uuid: "386ba12521e449c5acaccadee93e3725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } } }
I20260810 21:01:15.368762 27521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [sys.catalog]: This master's current role is: LEADER
I20260810 21:01:15.369091 27516 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 21:01:15.374610 27525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386ba12521e449c5acaccadee93e3725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ba12521e449c5acaccadee93e3725" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 37387 } } }
I20260810 21:01:15.374943 27525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725 [sys.catalog]: This master's current role is: LEADER
I20260810 21:01:15.376391 27535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:01:15.377895 27535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:01:15.383818 27535 catalog_manager.cc:1347] Generated new cluster ID: e492d2d8236b482a9c6ba1487ca95134
I20260810 21:01:15.383872 27535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:01:15.392390 27535 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:01:15.393093 27535 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:01:15.404171 27535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: Generated new TSK 0
I20260810 21:01:15.404422 27535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:01:16.264642 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:35565/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:01:16.383878 27565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:01:16.384101 27565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:01:16.384135 27565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:01:16.385308 27565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:01:16.385344 27565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:01:16.385370 27565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:01:16.386740 27565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.200.20:35565/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:01:16.386941 27565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:01:16.387183 27565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:01:16.394273 27571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:16.394277 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:16.394490 27565 server_base.cc:1034] running on GCE node
W20260810 21:01:16.394304 27573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:16.394759 27565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:01:16.395001 27565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:01:16.396157 27565 hybrid_clock.cc:648] HybridClock initialized: now 1786395676396144 us; error 28 us; skew 500 ppm
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:16.399235 27565 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:01:16.399847 27565 webserver.cc:460] Webserver started at http://127.14.200.1:46209/ using document root <none> and password file <none>
I20260810 21:01:16.400097 27565 fs_manager.cc:359] Metadata directory not provided
I20260810 21:01:16.400188 27565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:01:16.400317 27565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:16.417860 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "db39d9b13d834e6195664ee1480b3221"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "4cd644c6a2a3b605f4e8fb6df91b8c2f"
server_key_iv: "8968e477cc70d44e3cb635920c3074b1"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.418807 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "db39d9b13d834e6195664ee1480b3221"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "4cd644c6a2a3b605f4e8fb6df91b8c2f"
server_key_iv: "8968e477cc70d44e3cb635920c3074b1"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.421197 27565 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 21:01:16.437826 27581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.438068 27565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 21:01:16.438160 27565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "db39d9b13d834e6195664ee1480b3221"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "4cd644c6a2a3b605f4e8fb6df91b8c2f"
server_key_iv: "8968e477cc70d44e3cb635920c3074b1"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.438248 27565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:01:16.465724 27565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:01:16.469105 27565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:01:16.469281 27565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:01:16.469528 27565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:01:16.469904 27565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:01:16.469959 27565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.470003 27565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:01:16.470028 27565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.489650 27565 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:37417
I20260810 21:01:16.490247 27565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:01:16.493053 27705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:37417 every 8 connection(s)
I20260810 21:01:16.497545 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 27565
I20260810 21:01:16.497694 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:01:16.497925 15136 external_mini_cluster.cc:1428] Setting key 66fc6eec88899c2fdec2d147d331a605
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:01:16.507783 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:16.491247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:50437 (local address 127.14.200.62:37387)
0810 21:01:16.493441 (+ 2194us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:16.493444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:16.493454 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:01:16.493482 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:16.493483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:16.493496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:16.493570 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:16.496378 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.497306 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.499249 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.499582 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.499672 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:16.504733 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:16.504735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:16.504735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:16.504742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:16.506390 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:16.506793 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:16.506794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:16.506795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:16.506855 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:16.507129 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:16.507130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:16.507130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:16.507246 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:16.507314 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:16.507602 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 21:01:16.507691 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2170,"thread_start_us":146,"threads_started":1}
I20260810 21:01:16.507951 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:16.491286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:37387 (local address 127.14.200.1:50437)
0810 21:01:16.491615 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:16.491665 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:16.492549 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:16.493743 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:16.493748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:16.495780 (+ 2032us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:01:16.496243 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:16.496251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.497425 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.497426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:16.499140 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:16.499143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.499713 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.499715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:16.500043 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:16.502790 (+ 2747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:01:16.502802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:01:16.504628 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:01:16.506473 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:16.506481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:16.506483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:16.506703 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:16.506948 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:16.506949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:16.506950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:16.507062 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:16.507319 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:01:16.507326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:01:16.507466 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 21:01:16.507569 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 21:01:16.507712 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":225,"threads_started":1}
I20260810 21:01:16.508533 27706 heartbeater.cc:347] Connected to a master server at 127.14.200.62:37387
I20260810 21:01:16.508648 27706 heartbeater.cc:464] Registering TS with master...
I20260810 21:01:16.508940 27706 heartbeater.cc:511] Master 127.14.200.62:37387 requested a full tablet report, sending...
I20260810 21:01:16.509473 27479 ts_manager.cc:194] Registered new tserver with Master: db39d9b13d834e6195664ee1480b3221 (127.14.200.1:37417)
I20260810 21:01:16.510058 27479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:50437
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:01:16.536365 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:35565/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:01:16.673571 27714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:01:16.673825 27714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:01:16.673862 27714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:01:16.675438 27714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:01:16.675493 27714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:01:16.675531 27714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:01:16.677747 27714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.200.20:35565/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:01:16.677986 27714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:01:16.678260 27714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 21:01:16.685649 27724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:16.685778 27722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:16.685896 27726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:16.686738 27714 server_base.cc:1034] running on GCE node
I20260810 21:01:16.686939 27714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:01:16.687234 27714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:01:16.688408 27714 hybrid_clock.cc:648] HybridClock initialized: now 1786395676688383 us; error 36 us; skew 500 ppm
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:16.692021 27714 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:01:16.692677 27714 webserver.cc:460] Webserver started at http://127.14.200.2:43285/ using document root <none> and password file <none>
I20260810 21:01:16.693048 27714 fs_manager.cc:359] Metadata directory not provided
I20260810 21:01:16.693194 27714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:01:16.693365 27714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:16.715292 27714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "1ad54aaa8d324174b13d2d62e45f5715"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "ee0b5f569de25c88768ebd70d625bd15"
server_key_iv: "b5e1d7b5bf9a19075b2b977ecfe07426"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.715858 27714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "1ad54aaa8d324174b13d2d62e45f5715"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "ee0b5f569de25c88768ebd70d625bd15"
server_key_iv: "b5e1d7b5bf9a19075b2b977ecfe07426"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.718473 27714 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 21:01:16.742609 27733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.742866 27714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 21:01:16.742973 27714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "1ad54aaa8d324174b13d2d62e45f5715"
format_stamp: "Formatted at 2026-08-10 21:01:16 on dist-test-slave-1x4g"
server_key: "ee0b5f569de25c88768ebd70d625bd15"
server_key_iv: "b5e1d7b5bf9a19075b2b977ecfe07426"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:16.743070 27714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:01:16.769513 27714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:01:16.772207 27714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:01:16.772408 27714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:01:16.772759 27714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:01:16.773365 27714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:01:16.773478 27714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.773561 27714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:01:16.773623 27714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:16.797037 27714 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:44797
I20260810 21:01:16.797567 27714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:01:16.805857 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 27714
I20260810 21:01:16.805969 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:01:16.806172 15136 external_mini_cluster.cc:1428] Setting key c421757cb7c876a25ca4975afc0f973f
I20260810 21:01:16.798804 27856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:44797 every 8 connection(s)
Aug 10 21:01:16 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:01:16.819097 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:16.799778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:42449 (local address 127.14.200.62:37387)
0810 21:01:16.802497 (+ 2719us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:16.802500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:16.802513 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:01:16.802554 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:16.802555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:16.802569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:16.802647 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:16.803586 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.804423 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.811466 (+ 7043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.811766 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.811817 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:16.815064 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:16.815066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:16.815066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:16.815074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:16.817097 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:16.817750 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:16.817751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:16.817751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:16.817817 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:16.818181 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:16.818182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:16.818183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:16.818332 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:16.818409 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:16.818890 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 21:01:16.819002 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2687}
I20260810 21:01:16.819685 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:16.799427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:37387 (local address 127.14.200.2:42449)
0810 21:01:16.800056 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:16.800103 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:16.800756 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:16.802770 (+ 2014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:16.802775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:16.803125 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:01:16.803440 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:16.803448 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.804563 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.804563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:16.811325 (+ 6762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:16.811329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:16.811920 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:16.811921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:16.812107 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:16.812901 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:01:16.812912 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:01:16.814862 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:01:16.817204 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:16.817212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:16.817215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:16.817615 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:16.817899 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:16.817900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:16.817900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:16.818079 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:16.818460 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:01:16.818467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:01:16.818661 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 21:01:16.819209 (+ 548us) client_negotiation.cc:240] Negotiation successful
0810 21:01:16.819415 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":143,"threads_started":1}
I20260810 21:01:16.820482 27857 heartbeater.cc:347] Connected to a master server at 127.14.200.62:37387
I20260810 21:01:16.820601 27857 heartbeater.cc:464] Registering TS with master...
I20260810 21:01:16.820844 27857 heartbeater.cc:511] Master 127.14.200.62:37387 requested a full tablet report, sending...
I20260810 21:01:16.821321 27479 ts_manager.cc:194] Registered new tserver with Master: 1ad54aaa8d324174b13d2d62e45f5715 (127.14.200.2:44797)
I20260810 21:01:16.821841 27479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:42449
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:01:16.859326 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:35565/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:01:17.003161 27862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:01:17.003440 27862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:01:17.003481 27862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:01:17.004642 27862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:01:17.004686 27862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:01:17.004712 27862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:01:17.006165 27862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.200.20:35565/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:37387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:01:17.006440 27862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:01:17.006706 27862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:01:17.011981 27872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:17.011996 27869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:17.012151 27862 server_base.cc:1034] running on GCE node
W20260810 21:01:17.012251 27868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:17.012584 27862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:01:17.012892 27862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:01:17.014078 27862 hybrid_clock.cc:648] HybridClock initialized: now 1786395677014065 us; error 30 us; skew 500 ppm
Aug 10 21:01:17 dist-test-slave-1x4g krb5kdc[26966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395677, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:17.016759 27862 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:01:17.017402 27862 webserver.cc:460] Webserver started at http://127.14.200.3:35851/ using document root <none> and password file <none>
I20260810 21:01:17.017620 27862 fs_manager.cc:359] Metadata directory not provided
I20260810 21:01:17.017688 27862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:01:17.017822 27862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:01:17 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395677, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:17.031021 27862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "18a8b9c686854e2dac268eb1fa27615d"
format_stamp: "Formatted at 2026-08-10 21:01:17 on dist-test-slave-1x4g"
server_key: "ba5c171b65ef9163b7930bdc47b5152a"
server_key_iv: "3c9109a3abc201ee0cf037ae584e5310"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:17.031430 27862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "18a8b9c686854e2dac268eb1fa27615d"
format_stamp: "Formatted at 2026-08-10 21:01:17 on dist-test-slave-1x4g"
server_key: "ba5c171b65ef9163b7930bdc47b5152a"
server_key_iv: "3c9109a3abc201ee0cf037ae584e5310"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:17.033164 27862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:01:17.056263 27879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:17.056515 27862 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260810 21:01:17.056672 27862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "18a8b9c686854e2dac268eb1fa27615d"
format_stamp: "Formatted at 2026-08-10 21:01:17 on dist-test-slave-1x4g"
server_key: "ba5c171b65ef9163b7930bdc47b5152a"
server_key_iv: "3c9109a3abc201ee0cf037ae584e5310"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:17.056851 27862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:01:17.072690 27862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:01:17.075337 27862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:01:17.075582 27862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:01:17.075901 27862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:01:17.076400 27862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:01:17.076506 27862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:17.076583 27862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:01:17.076691 27862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:17.091039 27862 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:38485
I20260810 21:01:17.091102 27997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:38485 every 8 connection(s)
I20260810 21:01:17.091607 27862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:01:17.098742 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 27862
I20260810 21:01:17.098870 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:01:17.099095 15136 external_mini_cluster.cc:1428] Setting key 90763d314fc5bb499db921f66d9f3f00
Aug 10 21:01:17 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395677, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:01:17.104611 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.092886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:60717 (local address 127.14.200.62:37387)
0810 21:01:17.092944 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.092946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.093114 (+ 168us) server_negotiation.cc:407] Connection header received
0810 21:01:17.093951 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.093953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.093969 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.094007 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:17.094956 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.095759 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.097324 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.097640 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.097728 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.101053 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:17.101054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:17.101055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:17.101063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:17.103011 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:17.103516 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:17.103517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:17.103518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:17.103579 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:17.103842 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:17.103843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:17.103843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:17.103973 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:17.104072 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.104422 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.104514 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 21:01:17.105497 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.092539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:37387 (local address 127.14.200.3:60717)
0810 21:01:17.092941 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.092994 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.093781 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.094223 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.094227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.094526 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:01:17.094815 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.094822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.095877 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.095878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.097220 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.097223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.097821 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.097822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.098004 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.098719 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:01:17.098730 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:01:17.100912 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:01:17.103109 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:17.103115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:17.103118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:17.103420 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:17.103647 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:01:17.103648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:01:17.103648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:01:17.103773 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:01:17.104064 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:01:17.104071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:01:17.104235 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.105081 (+ 846us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.105248 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":87,"threads_started":1}
I20260810 21:01:17.106135 27998 heartbeater.cc:347] Connected to a master server at 127.14.200.62:37387
I20260810 21:01:17.106230 27998 heartbeater.cc:464] Registering TS with master...
I20260810 21:01:17.106427 27998 heartbeater.cc:511] Master 127.14.200.62:37387 requested a full tablet report, sending...
I20260810 21:01:17.106853 27479 ts_manager.cc:194] Registered new tserver with Master: 18a8b9c686854e2dac268eb1fa27615d (127.14.200.3:38485)
I20260810 21:01:17.107317 27479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:60717
I20260810 21:01:17.111621 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:01:17.121541 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.113529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.14.200.62:37387)
0810 21:01:17.113568 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.113570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.113737 (+ 167us) server_negotiation.cc:407] Connection header received
0810 21:01:17.113790 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.113790 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.113803 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.113869 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:17.114582 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.115325 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.116402 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.116711 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.116816 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.118318 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:17.118319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:17.118320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:17.118328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:17.120102 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:17.120584 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:17.120586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:17.120586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:17.120652 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:17.120924 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:17.120925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:17.120925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:17.121058 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:17.121159 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.121349 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.121450 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:01:17.123294 27479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:01:17.125918 27479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:01:17.136174 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.130566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:38485 (local address 127.0.0.1:56234)
0810 21:01:17.131338 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.131349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.131427 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.131872 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.131876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.131878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:01:17.131958 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.131961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.133207 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.133208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.134535 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.134538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.135840 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.135841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.135932 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.135950 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.136040 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.136092 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":194,"threads_started":1}
I20260810 21:01:17.136245 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.130924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.14.200.3:38485)
0810 21:01:17.131621 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.131626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.131636 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:01:17.131678 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.131680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.131696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.131768 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:01:17.132050 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.133106 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.134770 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.135744 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.135827 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.136034 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.136063 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.136103 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":224,"threads_started":1}
I20260810 21:01:17.137248 27930 tablet_service.cc:1467] Processing CreateTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 (DEFAULT_TABLE table=test-table [id=0f9959306c414e7e861991a0dc180b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:01:17.137629 27930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b0ca397af4d9cb8ec2aae1a50ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:01:17.141261 28017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.130271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:44797 (local address 127.0.0.1:45938)
0810 21:01:17.130802 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.130812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.130854 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.136340 (+ 5486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.136341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.136344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:01:17.136397 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.136398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.138190 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.138191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.139660 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.139663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.141009 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.141010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.141084 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.141089 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.141145 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.141187 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":180,"threads_started":1}
I20260810 21:01:17.141786 28016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.130373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.14.200.2:44797)
0810 21:01:17.135987 (+ 5614us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.135992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.136015 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:01:17.136067 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.136069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.136085 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.136168 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:01:17.136844 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.138083 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.139967 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.140755 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.140847 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.140967 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.141650 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.141691 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5531,"thread_start_us":109,"threads_started":1}
I20260810 21:01:17.142261 28019 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Bootstrap starting.
I20260810 21:01:17.142639 27769 tablet_service.cc:1467] Processing CreateTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 (DEFAULT_TABLE table=test-table [id=0f9959306c414e7e861991a0dc180b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:01:17.143090 27769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b0ca397af4d9cb8ec2aae1a50ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:01:17.143957 28019 tablet_bootstrap.cc:654] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Neither blocks nor log segments found. Creating new log.
I20260810 21:01:17.144520 28019 log.cc:824] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Log is configured to *not* fsync() on all Append() calls
I20260810 21:01:17.145454 28019 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: No bootstrap required, opened a new log
I20260810 21:01:17.145514 28019 ts_tablet_manager.cc:1397] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 21:01:17.148021 28019 raft_consensus.cc:348] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.148183 28019 raft_consensus.cc:374] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:01:17.148211 28019 raft_consensus.cc:729] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a8b9c686854e2dac268eb1fa27615d, State: Initialized, Role: FOLLOWER
I20260810 21:01:17.148469 28019 consensus_queue.cc:260] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.148878 28019 ts_tablet_manager.cc:1428] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 21:01:17.149041 27998 heartbeater.cc:503] Master 127.14.200.62:37387 was elected leader, sending a full tablet report...
I20260810 21:01:17.152904 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.129856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:37417 (local address 127.0.0.1:59148)
0810 21:01:17.130536 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.130578 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.130650 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.148156 (+ 17506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.148158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.148160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:01:17.148209 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.148211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.149960 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.149961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.151127 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.151131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.152662 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.152663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.152729 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.152734 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.152765 (+ 31us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.152827 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":217,"threads_started":1}
I20260810 21:01:17.152937 28022 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: Bootstrap starting.
I20260810 21:01:17.153298 28013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.129965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.14.200.1:37417)
0810 21:01:17.144329 (+ 14364us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.144345 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.144358 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:01:17.144395 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.144397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.144415 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.144515 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:01:17.148345 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.149846 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.151309 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.152563 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.153089 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.153160 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.153187 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.153229 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14323,"thread_start_us":164,"threads_started":1}
I20260810 21:01:17.153991 27626 tablet_service.cc:1467] Processing CreateTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 (DEFAULT_TABLE table=test-table [id=0f9959306c414e7e861991a0dc180b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:01:17.154225 28022 tablet_bootstrap.cc:654] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: Neither blocks nor log segments found. Creating new log.
I20260810 21:01:17.154345 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d23b0ca397af4d9cb8ec2aae1a50ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:01:17.154568 28022 log.cc:824] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: Log is configured to *not* fsync() on all Append() calls
I20260810 21:01:17.155521 28022 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: No bootstrap required, opened a new log
I20260810 21:01:17.155594 28022 ts_tablet_manager.cc:1397] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 21:01:17.158291 28022 raft_consensus.cc:348] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.158514 28026 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Bootstrap starting.
I20260810 21:01:17.158633 28022 raft_consensus.cc:374] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:01:17.158777 28022 raft_consensus.cc:729] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad54aaa8d324174b13d2d62e45f5715, State: Initialized, Role: FOLLOWER
I20260810 21:01:17.159076 28022 consensus_queue.cc:260] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.159389 28022 ts_tablet_manager.cc:1428] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 21:01:17.159442 28026 tablet_bootstrap.cc:654] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Neither blocks nor log segments found. Creating new log.
I20260810 21:01:17.159547 27857 heartbeater.cc:503] Master 127.14.200.62:37387 was elected leader, sending a full tablet report...
I20260810 21:01:17.160836 28026 log.cc:824] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Log is configured to *not* fsync() on all Append() calls
I20260810 21:01:17.161670 28026 tablet_bootstrap.cc:492] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: No bootstrap required, opened a new log
I20260810 21:01:17.161731 28026 ts_tablet_manager.cc:1397] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 21:01:17.165246 28026 raft_consensus.cc:348] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.165398 28026 raft_consensus.cc:374] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:01:17.165434 28026 raft_consensus.cc:729] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db39d9b13d834e6195664ee1480b3221, State: Initialized, Role: FOLLOWER
I20260810 21:01:17.165700 28026 consensus_queue.cc:260] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.166071 28026 ts_tablet_manager.cc:1428] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260810 21:01:17.166232 27706 heartbeater.cc:503] Master 127.14.200.62:37387 was elected leader, sending a full tablet report...
Aug 10 21:01:17 dist-test-slave-1x4g krb5kdc[26966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395676, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
W20260810 21:01:17.243880 27708 tablet.cc:2369] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:01:17.258927 28024 raft_consensus.cc:482] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:01:17.259107 28024 raft_consensus.cc:504] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.259608 28024 leader_election.cc:288] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db39d9b13d834e6195664ee1480b3221 (127.14.200.1:37417), 1ad54aaa8d324174b13d2d62e45f5715 (127.14.200.2:44797)
I20260810 21:01:17.264022 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.259786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:37417 (local address 127.14.200.3:46345)
0810 21:01:17.259832 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.259846 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.259893 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.260130 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.260131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.260133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:01:17.260345 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.260348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.261575 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.261576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.262683 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.262686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.263769 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.263771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.263840 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.263843 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.263886 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.263937 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 21:01:17.264513 28013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.259922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:46345 (local address 127.14.200.1:37417)
0810 21:01:17.259974 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.259976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.259984 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:01:17.260014 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.260015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.260028 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.260078 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:01:17.260461 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.261414 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.262879 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.263655 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.263728 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.263738 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.264404 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.264450 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 21:01:17.264901 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b0ca397af4d9cb8ec2aae1a50ff71" candidate_uuid: "18a8b9c686854e2dac268eb1fa27615d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db39d9b13d834e6195664ee1480b3221" is_pre_election: true
I20260810 21:01:17.265184 27655 raft_consensus.cc:2454] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a8b9c686854e2dac268eb1fa27615d in term 0.
I20260810 21:01:17.265269 28016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.260019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:35395 (local address 127.14.200.2:44797)
0810 21:01:17.260058 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.260060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.260762 (+ 702us) server_negotiation.cc:407] Connection header received
0810 21:01:17.260920 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.260921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.260933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.260970 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:01:17.261515 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.262185 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.264004 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.264656 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.264711 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.264720 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.265147 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.265197 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:01:17.265573 27883 leader_election.cc:302] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a8b9c686854e2dac268eb1fa27615d, db39d9b13d834e6195664ee1480b3221; no voters:
I20260810 21:01:17.265699 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.259840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:44797 (local address 127.14.200.3:35395)
0810 21:01:17.260678 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.260692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.260819 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.261048 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:01:17.261048 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:01:17.261050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:01:17.261390 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.261394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.262448 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.262449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.263883 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:01:17.263887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.264926 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.264928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:01:17.265032 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.265037 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:01:17.265407 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 21:01:17.265453 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":185,"threads_started":1}
I20260810 21:01:17.265759 28024 raft_consensus.cc:2781] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:01:17.265826 28024 raft_consensus.cc:482] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:01:17.265848 28024 raft_consensus.cc:3037] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:01:17.267099 28024 raft_consensus.cc:504] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.267247 28024 leader_election.cc:288] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [CANDIDATE]: Term 1 election: Requested vote from peers db39d9b13d834e6195664ee1480b3221 (127.14.200.1:37417), 1ad54aaa8d324174b13d2d62e45f5715 (127.14.200.2:44797)
I20260810 21:01:17.267428 27804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b0ca397af4d9cb8ec2aae1a50ff71" candidate_uuid: "18a8b9c686854e2dac268eb1fa27615d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad54aaa8d324174b13d2d62e45f5715" is_pre_election: true
I20260810 21:01:17.267501 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b0ca397af4d9cb8ec2aae1a50ff71" candidate_uuid: "18a8b9c686854e2dac268eb1fa27615d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db39d9b13d834e6195664ee1480b3221"
I20260810 21:01:17.267560 27804 raft_consensus.cc:2454] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a8b9c686854e2dac268eb1fa27615d in term 0.
I20260810 21:01:17.267590 27655 raft_consensus.cc:3037] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:01:17.267709 27805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d23b0ca397af4d9cb8ec2aae1a50ff71" candidate_uuid: "18a8b9c686854e2dac268eb1fa27615d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad54aaa8d324174b13d2d62e45f5715"
I20260810 21:01:17.267763 27805 raft_consensus.cc:3037] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:01:17.268908 27805 raft_consensus.cc:2454] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a8b9c686854e2dac268eb1fa27615d in term 1.
I20260810 21:01:17.269161 27881 leader_election.cc:302] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a8b9c686854e2dac268eb1fa27615d, 1ad54aaa8d324174b13d2d62e45f5715; no voters:
I20260810 21:01:17.269167 27655 raft_consensus.cc:2454] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a8b9c686854e2dac268eb1fa27615d in term 1.
I20260810 21:01:17.269251 28024 raft_consensus.cc:2781] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:01:17.269372 28024 raft_consensus.cc:686] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 1 LEADER]: Becoming Leader. State: Replica: 18a8b9c686854e2dac268eb1fa27615d, State: Running, Role: LEADER
I20260810 21:01:17.269454 28024 consensus_queue.cc:237] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } }
I20260810 21:01:17.270208 27479 catalog_manager.cc:5495] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a8b9c686854e2dac268eb1fa27615d (127.14.200.3). New cstate: current_term: 1 leader_uuid: "18a8b9c686854e2dac268eb1fa27615d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18a8b9c686854e2dac268eb1fa27615d" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260810 21:01:17.285563 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.282012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.14.200.3:38485)
0810 21:01:17.282072 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:17.282074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:17.282116 (+ 42us) server_negotiation.cc:407] Connection header received
0810 21:01:17.282203 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:17.282204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:17.282223 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:17.282307 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:01:17.282489 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.283197 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.284373 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:17.284662 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:17.284725 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:17.285002 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:01:17.285119 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:01:17.285198 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:17.285472 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 21:01:17.285494 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:01:17.288089 27805 raft_consensus.cc:1261] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 1 FOLLOWER]: Refusing update from remote peer 18a8b9c686854e2dac268eb1fa27615d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:01:17.288462 28024 consensus_queue.cc:1035] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad54aaa8d324174b13d2d62e45f5715" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:01:17.290917 27655 raft_consensus.cc:1261] T d23b0ca397af4d9cb8ec2aae1a50ff71 P db39d9b13d834e6195664ee1480b3221 [term 1 FOLLOWER]: Refusing update from remote peer 18a8b9c686854e2dac268eb1fa27615d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:01:17.291260 28024 consensus_queue.cc:1035] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db39d9b13d834e6195664ee1480b3221" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:01:17.295941 28034 mvcc.cc:205] Tried to move back new op lower bound from 7317076694169133056 to 7317076694096207872. Current Snapshot: MvccSnapshot[applied={T|T < 7317076694169133056}]
I20260810 21:01:17.296165 28037 mvcc.cc:205] Tried to move back new op lower bound from 7317076694169133056 to 7317076694096207872. Current Snapshot: MvccSnapshot[applied={T|T < 7317076694169133056}]
I20260810 21:01:17.298940 28035 mvcc.cc:205] Tried to move back new op lower bound from 7317076694169133056 to 7317076694096207872. Current Snapshot: MvccSnapshot[applied={T|T < 7317076694169133056}]
I20260810 21:01:17.302098 27479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:01:17.302181 27479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:01:17.302894 27479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 386ba12521e449c5acaccadee93e3725: Sending DeleteTablet for 3 replicas of tablet d23b0ca397af4d9cb8ec2aae1a50ff71
I20260810 21:01:17.303303 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:01:17 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:59148
I20260810 21:01:17.303603 27769 tablet_service.cc:1514] Processing DeleteTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:01:17 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:45938
I20260810 21:01:17.304057 27930 tablet_service.cc:1514] Processing DeleteTablet for tablet d23b0ca397af4d9cb8ec2aae1a50ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:01:17 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:56234
I20260810 21:01:17.304461 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 27565
I20260810 21:01:17.311048 28045 tablet_replica.cc:330] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: stopping tablet replica
I20260810 21:01:17.311209 28045 raft_consensus.cc:2229] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:01:17.311367 28045 raft_consensus.cc:2258] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:01:17.311678 28044 tablet_replica.cc:330] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715: stopping tablet replica
I20260810 21:01:17.311841 28044 raft_consensus.cc:2229] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:01:17.311964 28044 raft_consensus.cc:2258] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 1ad54aaa8d324174b13d2d62e45f5715 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 21:01:17.316977 27463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:37417 (error 108)
I20260810 21:01:17.321020 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 27714
I20260810 21:01:17.322255 28045 ts_tablet_manager.cc:1905] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:01:17.327181 28045 ts_tablet_manager.cc:1918] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:01:17.327275 28045 log.cc:1196] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/d23b0ca397af4d9cb8ec2aae1a50ff71
I20260810 21:01:17.327564 28045 ts_tablet_manager.cc:1939] T d23b0ca397af4d9cb8ec2aae1a50ff71 P 18a8b9c686854e2dac268eb1fa27615d: Deleting consensus metadata
I20260810 21:01:17.328015 27460 catalog_manager.cc:4841] TS 18a8b9c686854e2dac268eb1fa27615d (127.14.200.3:38485): tablet d23b0ca397af4d9cb8ec2aae1a50ff71 (table test-table [id=0f9959306c414e7e861991a0dc180b02]) successfully deleted
I20260810 21:01:17.337334 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:17.337148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:37417 (local address 127.0.0.1:59162)
0810 21:01:17.337212 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.337289 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 21:01:17.337554 27463 catalog_manager.cc:4584] TS db39d9b13d834e6195664ee1480b3221 (127.14.200.1:37417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d23b0ca397af4d9cb8ec2aae1a50ff71: Network error: Client connection negotiation failed: client connection to 127.14.200.1:37417: connect: Connection refused (error 111)
W20260810 21:01:17.350376 28014 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:01:17.350078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:44797 (local address 127.0.0.1:45942)
0810 21:01:17.350160 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 21:01:17.350175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 21:01:17.350260 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:01:17.350322 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.2:44797: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 21:01:17.350658 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 27862
I20260810 21:01:17.358914 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 27443
I20260810 21:01:17.367599 15136 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 21:01:17.711647 15136 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 21:01:17.711742 15136 mini_ranger.cc:66] Stopping Ranger...
I20260810 21:01:18.055711 15136 mini_ranger.cc:68] Stopped Ranger
2026-08-10 21:01:18.055 UTC [26998] LOG: received smart shutdown request
2026-08-10 21:01:18.059 UTC [26998] LOG: background worker "logical replication launcher" (PID 27007) exited with exit code 1
2026-08-10 21:01:18.059 UTC [27002] LOG: shutting down
2026-08-10 21:01:18.209 UTC [26998] LOG: database system is shut down
2026-08-10T21:01:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47427 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:01:18 dist-test-slave-1x4g krb5kdc[28055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:01:18 dist-test-slave-1x4g krb5kdc[28055](info): set up 2 sockets
Aug 10 21:01:18 dist-test-slave-1x4g krb5kdc[28055](info): commencing operation
krb5kdc: starting...
W20260810 21:01:19.360419 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:01:19 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:01:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangeradmin_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangeradmin_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangerlookup_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangerlookup_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
I20260810 21:01:19.515492 15136 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrRlFwH/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/postgres -l logfile start
2026-08-10 21:01:21.819 UTC [28086] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 21:01:21.819 UTC [28086] LOG: listening on IPv4 address "127.14.200.20", port 40821
2026-08-10 21:01:21.822 UTC [28086] LOG: listening on Unix socket "/tmp/.s.PGSQL.40821"
2026-08-10 21:01:21.842 UTC [28089] LOG: database system was shut down at 2026-08-10 21:01:20 UTC
2026-08-10 21:01:21.846 UTC [28086] LOG: database system is ready to accept connections
I20260810 21:01:22.821724 15136 mini_postgres.cc:96] Postgres bound to 40821
2026-08-10 21:01:22.829 UTC [28098] FATAL: database "slave" does not exist
127.14.200.20:40821 - accepting connections
I20260810 21:01:22.829710 15136 mini_ranger.cc:161] Starting Ranger...
I20260810 21:01:22.847143 15136 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 21:01:23.274463 15136 mini_ranger.cc:87] Created ranger Postgres database
I20260810 21:01:23.274557 15136 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin
2026-08-10 21:01:23,531 [I] DB FLAVOR :POSTGRES
2026-08-10 21:01:23,532 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 21:01:23,532 [I] Checking connection..
2026-08-10 21:01:23,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 21:01:23,749 [I] Checking connection passed.
2026-08-10 21:01:23,749 [I] --------- Verifying version history table ---------
2026-08-10 21:01:23,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:01:23,971 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 21:01:23,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:01:24,216 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 21:01:24,217 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 21:01:24,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 21:01:24.441 UTC [28165] WARNING: there is no transaction in progress
2026-08-10 21:01:24,447 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 21:01:24,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 21:01:24,691 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 21:01:24,691 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 21:01:24,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 21:01:24,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 21:01:25,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-10 21:01:25,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 21:01:25,633 [I] Table x_portal_user does not exist in database ranger
2026-08-10 21:01:25,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 21:01:25,881 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 21:01:25,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 21:01:26,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 21:01:26,389 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 21:01:26,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 21:01:27.264 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.279 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.294 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.307 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.321 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.337 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.387 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.393 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.405 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.415 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.424 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.437 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.446 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.451 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.457 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:27.463 UTC [28283] WARNING: there is no transaction in progress
2026-08-10 21:01:28,021 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 21:01:28,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 21:01:28,260 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 21:01:28,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 21:01:28,494 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 21:01:28,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 21:01:28,718 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 21:01:28,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 21:01:28,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 21:01:29,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-10 21:01:29,418 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 21:01:29,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 21:01:29,664 [I] DB_PATCHES have already been applied
I20260810 21:01:29.670305 15136 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrRlFwH/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 21:01:29.670396 15136 mini_ranger.cc:193] Using host: 127.14.200.20
I20260810 21:01:29.674242 15136 mini_ranger.cc:227] Ranger admin URL: http://127.14.200.20:49061
Aug 10, 2026 9:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 9:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 9:01:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 9:01:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 9:01:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 9:01:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 9:01:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 21:01:30.804418 15136 mini_ranger.cc:160] Time spent starting Ranger: real 7.975s user 0.003s sys 0.008s
Aug 10, 2026 9:01:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 9:01:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 9:01:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 9:01:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 9:01:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 21:01:51.845388 15136 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.200.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.200.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangerkms_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.200.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/rangerkms_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.200.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.200.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.200.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/HTTP_127.14.200.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 21:01:51 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395711, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 21:01:51.926498 15136 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 21:01:51.934937 15136 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 21:01:52.702541 15136 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 21:01:52.702630 15136 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms
I20260810 21:01:52.702653 15136 mini_ranger_kms.cc:227] Using postgres at 127.14.200.20:40821
2026-08-10 21:01:52,871 [I] DB FLAVOR :POSTGRES
2026-08-10 21:01:52,871 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 21:01:52,871 [I] Checking connection
2026-08-10 21:01:52,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 21:01:53,095 [I] connection success
2026-08-10 21:01:53,095 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 21:01:53,095 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 21:01:53,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 21:01:53,341 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 21:01:53,341 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 21:01:53,342 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 21:01:53,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.200.20:40821/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 21:01:53,581 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 21:01:54.001044 15136 mini_ranger_kms.cc:310] Created kms service
I20260810 21:01:54.194473 15136 mini_ranger_kms.cc:326] Created kudu user
I20260810 21:01:54.239181 15136 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 21:01:54 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395711, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:54.627492 15136 mini_ranger_kms.cc:384] Added ranger policy
I20260810 21:01:54.627609 15136 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrRlFwH/build/release/bin/postgresql.jar:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrRlFwH/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 21:01:54.627624 15136 mini_ranger_kms.cc:240] Using host: 127.14.200.20
I20260810 21:01:54.631597 15136 mini_ranger_kms.cc:276] Ranger KMS PID: 28477
I20260810 21:01:54.631668 15136 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.200.20:56727
Aug 10, 2026 9:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 9:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 9:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 9:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 9:01:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 9:01:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 21:01:56 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786395716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:56.882994 15136 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.956s user 0.000s sys 0.153s
I20260810 21:01:56.883116 15136 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 21:01:56.883170 15136 mini_ranger_kms.cc:401] 127.14.200.20:56727/kms/v1/keys
Aug 10 21:01:57 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
Aug 10 21:01:57 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.200.20@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
Aug 10 21:01:59 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:01:59.721799 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41369
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41369
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:01:59.804929 28532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:01:59.805114 28532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:01:59.805135 28532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:01:59.806298 28532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:01:59.806340 28532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:01:59.806354 28532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:01:59.806411 28532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:01:59.806427 28532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:01:59.807955 28532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.200.20:56727/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41369
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:01:59.808192 28532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:01:59.808405 28532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:01:59.813395 28537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:59.813496 28532 server_base.cc:1034] running on GCE node
W20260810 21:01:59.813442 28539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:01:59.813428 28541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:01:59.813872 28532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:01:59.814165 28532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:01:59.815303 28532 hybrid_clock.cc:648] HybridClock initialized: now 1786395719815283 us; error 40 us; skew 500 ppm
Aug 10 21:01:59 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395719, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:01:59.822876 28532 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:01:59.823292 28532 webserver.cc:460] Webserver started at http://127.14.200.62:37187/ using document root <none> and password file <none>
I20260810 21:01:59.823515 28532 fs_manager.cc:359] Metadata directory not provided
I20260810 21:01:59.823557 28532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:01:59.823709 28532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:01:59 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395719, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:01:59.882923 28532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "a103532b8da943e8a45d9c1dcc3e48a1"
format_stamp: "Formatted at 2026-08-10 21:01:59 on dist-test-slave-1x4g"
server_key: "071a311c215ce88c009df463c571d2b3"
server_key_iv: "137bdb3650ce64a95daef6be82fe29bd"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:59.883319 28532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "a103532b8da943e8a45d9c1dcc3e48a1"
format_stamp: "Formatted at 2026-08-10 21:01:59 on dist-test-slave-1x4g"
server_key: "071a311c215ce88c009df463c571d2b3"
server_key_iv: "137bdb3650ce64a95daef6be82fe29bd"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:59.884917 28532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:01:59.897874 28548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:01:59.898137 28532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:01:59.898310 28532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "a103532b8da943e8a45d9c1dcc3e48a1"
format_stamp: "Formatted at 2026-08-10 21:01:59 on dist-test-slave-1x4g"
server_key: "071a311c215ce88c009df463c571d2b3"
server_key_iv: "137bdb3650ce64a95daef6be82fe29bd"
server_key_version: "kuduclusterkey@0"
I20260810 21:01:59.898465 28532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:01:59.910759 28532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:01:59.912649 28532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:01:59.912858 28532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:01:59.917363 28532 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41369
I20260810 21:01:59.917459 28603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41369 every 8 connection(s)
I20260810 21:01:59.917788 28532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:01:59.918058 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 28532
I20260810 21:01:59.918177 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:01:59.918395 15136 external_mini_cluster.cc:1428] Setting key 2d301b360b76c2a62ab7de49ef5bf899
I20260810 21:01:59.918634 28605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:01:59.921487 28605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Bootstrap starting.
I20260810 21:01:59.922096 28605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Neither blocks nor log segments found. Creating new log.
I20260810 21:01:59.922387 28605 log.cc:824] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Log is configured to *not* fsync() on all Append() calls
I20260810 21:01:59.923151 28605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: No bootstrap required, opened a new log
Aug 10 21:01:59 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:01:59.925567 28605 raft_consensus.cc:348] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:01:59.925721 28605 raft_consensus.cc:374] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:01:59.925762 28605 raft_consensus.cc:729] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a103532b8da943e8a45d9c1dcc3e48a1, State: Initialized, Role: FOLLOWER
I20260810 21:01:59.925897 28605 consensus_queue.cc:260] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:01:59.925982 28605 raft_consensus.cc:388] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:01:59.926033 28605 raft_consensus.cc:482] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:01:59.926084 28605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:01:59.926713 28605 raft_consensus.cc:504] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:01:59.926877 28605 leader_election.cc:302] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a103532b8da943e8a45d9c1dcc3e48a1; no voters:
I20260810 21:01:59.927063 28605 leader_election.cc:288] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:01:59.927089 28610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:01:59.927194 28610 raft_consensus.cc:686] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 LEADER]: Becoming Leader. State: Replica: a103532b8da943e8a45d9c1dcc3e48a1, State: Running, Role: LEADER
I20260810 21:01:59.927337 28605 sys_catalog.cc:564] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:01:59.927345 28610 consensus_queue.cc:237] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:01:59.927861 28612 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a103532b8da943e8a45d9c1dcc3e48a1. Latest consensus state: current_term: 1 leader_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } } }
I20260810 21:01:59.928004 28612 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:01:59.928386 28611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } } }
I20260810 21:01:59.928570 28611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:01:59.928452 28620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:01:59.928895 28605 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 21:01:59.929263 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:01:59.918959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.14.200.62:41369)
0810 21:01:59.919141 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 21:01:59.919152 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 21:01:59.919186 (+ 34us) server_negotiation.cc:407] Connection header received
0810 21:01:59.919682 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:01:59.919688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:01:59.919732 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:01:59.919881 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:01:59.920492 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:59.921414 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:59.922235 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:01:59.922691 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:01:59.922834 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:01:59.925494 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:01:59.925499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:01:59.925500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:01:59.925521 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:01:59.927204 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:59.927647 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:59.927650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:59.927650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:59.927742 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:01:59.927963 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:01:59.927963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:01:59.927964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:01:59.928248 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:01:59.928855 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0810 21:01:59.928933 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 21:01:59.929067 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260810 21:01:59.929339 28620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:01:59.935122 28620 catalog_manager.cc:1347] Generated new cluster ID: 450878ac224f4e269a9e8738da98ee58
I20260810 21:01:59.935192 28620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:01:59.940863 28620 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:01:59.941573 28620 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:01:59.951901 28620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Generated new TSK 0
I20260810 21:01:59.952056 28620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:02:00.588292 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:00.706622 28651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:00.706893 28651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:00.706945 28651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:00.708562 28651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:00.708664 28651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:00.708714 28651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:00.710813 28651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:00.711072 28651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:00.711392 28651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:00.717620 28659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:00.717626 28658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:00.717986 28661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:00.718859 28651 server_base.cc:1034] running on GCE node
I20260810 21:02:00.719092 28651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:00.719368 28651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:00.720552 28651 hybrid_clock.cc:648] HybridClock initialized: now 1786395720720532 us; error 33 us; skew 500 ppm
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:00.723968 28651 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:00.724546 28651 webserver.cc:460] Webserver started at http://127.14.200.1:36323/ using document root <none> and password file <none>
I20260810 21:02:00.724875 28651 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:00.724943 28651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:00.725090 28651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:00.739131 28651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "3298154ecf8b474199c7b400cda2b641"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "3e84f287c486f736d3d07a90c1b01f97"
server_key_iv: "3df04bfd442e1262b6fbf1c70554c4b0"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.739661 28651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "3298154ecf8b474199c7b400cda2b641"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "3e84f287c486f736d3d07a90c1b01f97"
server_key_iv: "3df04bfd442e1262b6fbf1c70554c4b0"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.741788 28651 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:00.756419 28668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:00.756755 28651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260810 21:02:00.756891 28651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "3298154ecf8b474199c7b400cda2b641"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "3e84f287c486f736d3d07a90c1b01f97"
server_key_iv: "3df04bfd442e1262b6fbf1c70554c4b0"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.756985 28651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:00.770334 28651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:00.772768 28651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:00.772974 28651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:00.773204 28651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:00.773607 28651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:00.773655 28651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:00.773684 28651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:00.773734 28651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:00.781348 28651 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42789
I20260810 21:02:00.781781 28651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:00.784970 28785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42789 every 8 connection(s)
I20260810 21:02:00.785100 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 28651
I20260810 21:02:00.785203 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:02:00.785393 15136 external_mini_cluster.cc:1428] Setting key 14aed8adeeacdd1cf9fa50baeb9a35bd
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
I20260810 21:02:00.798283 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:00.782526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.1:44633)
0810 21:02:00.785777 (+ 3251us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:00.785827 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:00.788475 (+ 2648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:00.788811 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:00.788815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:00.789055 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:00.789324 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:00.789331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:00.790259 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:00.790260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:00.791494 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:00.791498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:00.792066 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:00.792067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:00.792211 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:00.792798 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:00.792809 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:00.794651 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:00.796544 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:00.796552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:00.796555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:00.796922 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:00.797173 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:00.797174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:00.797174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:00.797287 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:00.797567 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:00.797573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:00.797735 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 21:02:00.797840 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 21:02:00.797995 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2372,"thread_start_us":78,"threads_started":1}
I20260810 21:02:00.798585 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:00.782560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:44633 (local address 127.14.200.62:41369)
0810 21:02:00.785728 (+ 3168us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:00.785731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:00.788100 (+ 2369us) server_negotiation.cc:407] Connection header received
0810 21:02:00.788603 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:00.788604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:00.788619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:00.788700 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:00.789440 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:00.790152 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:00.791599 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:00.791892 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:00.791975 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:00.794768 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:00.794769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:00.794770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:00.794777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:00.796451 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:00.797020 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:00.797022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:00.797022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:00.797094 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:00.797361 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:00.797362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:00.797362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:00.797496 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:00.797634 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:00.797873 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 21:02:00.797961 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3133,"thread_start_us":98,"threads_started":1}
I20260810 21:02:00.800565 28786 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:00.800668 28786 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:00.800890 28786 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:00.801383 28567 ts_manager.cc:194] Registered new tserver with Master: 3298154ecf8b474199c7b400cda2b641 (127.14.200.1:42789)
I20260810 21:02:00.801985 28567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:44633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:02:00.815560 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:00.933002 28793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:00.933211 28793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:00.933250 28793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:00.934973 28793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:00.935055 28793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:00.935099 28793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:00.936622 28793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:00.936904 28793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:00.937183 28793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:00.943367 28806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:00.943383 28804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:00.943686 28803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:00.944751 28793 server_base.cc:1034] running on GCE node
I20260810 21:02:00.944976 28793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:00.945221 28793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:00.946422 28793 hybrid_clock.cc:648] HybridClock initialized: now 1786395720946403 us; error 33 us; skew 500 ppm
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:00.949563 28793 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:00.950049 28793 webserver.cc:460] Webserver started at http://127.14.200.2:36749/ using document root <none> and password file <none>
I20260810 21:02:00.950314 28793 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:00.950363 28793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:00.950497 28793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:02:00 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:00.969427 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "037e57b48a924e11bbc04f4b15ef2b28"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "5174d71a40148a4f0436a4c5ba9af711"
server_key_iv: "f0b9c375ad5610e10561aea9f9b95884"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.969923 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "037e57b48a924e11bbc04f4b15ef2b28"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "5174d71a40148a4f0436a4c5ba9af711"
server_key_iv: "f0b9c375ad5610e10561aea9f9b95884"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.973171 28793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 21:02:00.995129 28813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:00.995400 28793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:00.995515 28793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "037e57b48a924e11bbc04f4b15ef2b28"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "5174d71a40148a4f0436a4c5ba9af711"
server_key_iv: "f0b9c375ad5610e10561aea9f9b95884"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:00.995615 28793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:01.020190 28793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:01.023775 28793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:01.023952 28793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:01.024220 28793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:01.024760 28793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:01.024847 28793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.024912 28793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:01.024953 28793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.032377 28793 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:42101
I20260810 21:02:01.032464 28931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:42101 every 8 connection(s)
I20260810 21:02:01.033061 28793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395720, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:01.043079 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 28793
I20260810 21:02:01.043196 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:02:01.043445 15136 external_mini_cluster.cc:1428] Setting key 7b5efd306a3ea0652e1c8eef90b0dd3b
I20260810 21:02:01.044947 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.033988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:32999 (local address 127.14.200.62:41369)
0810 21:02:01.034040 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:01.034042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:01.034320 (+ 278us) server_negotiation.cc:407] Connection header received
0810 21:02:01.035037 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:01.035039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:01.035056 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:01.035087 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:01.035976 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.036815 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.038318 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.038522 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.038600 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.040948 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:01.040949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:01.040950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:01.040959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:01.042715 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.043258 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.043259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.043259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.043323 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.043631 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.043633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.043633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.043770 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:01.043816 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:01.044672 (+ 856us) server_negotiation.cc:299] Negotiation successful
0810 21:02:01.044768 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:02:01.045511 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.033975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.2:32999)
0810 21:02:01.034183 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:01.034231 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:01.034908 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:01.035275 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:01.035279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:01.035537 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:01.035808 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.035816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.036939 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.036940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.038162 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.038165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.038635 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.038636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.038754 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.039286 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:01.039295 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:01.040821 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:01.042815 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.042822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.042824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.043153 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.043402 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.043402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.043403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.043539 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.044319 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:01.044326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:01.044498 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 21:02:01.045073 (+ 575us) client_negotiation.cc:240] Negotiation successful
0810 21:02:01.045250 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260810 21:02:01.045923 28932 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:01.046008 28932 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.046208 28932 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:01.046566 28567 ts_manager.cc:194] Registered new tserver with Master: 037e57b48a924e11bbc04f4b15ef2b28 (127.14.200.2:42101)
I20260810 21:02:01.047021 28567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:32999
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:02:01.083696 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:01.192431 28937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:01.192808 28937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:01.192910 28937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:01.194700 28937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:01.194762 28937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:01.194803 28937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:01.196645 28937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:01.196964 28937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:01.197255 28937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:01.203891 28944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.204244 28943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.204450 28946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:01.205511 28937 server_base.cc:1034] running on GCE node
I20260810 21:02:01.205730 28937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:01.206005 28937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:01.207175 28937 hybrid_clock.cc:648] HybridClock initialized: now 1786395721207151 us; error 35 us; skew 500 ppm
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:01.210547 28937 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:01.211081 28937 webserver.cc:460] Webserver started at http://127.14.200.3:38225/ using document root <none> and password file <none>
I20260810 21:02:01.211349 28937 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:01.211401 28937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:01.211527 28937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:01.228605 28937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "28911e71554f408e9a0eb561fcd90ec9"
format_stamp: "Formatted at 2026-08-10 21:02:01 on dist-test-slave-1x4g"
server_key: "3a850baae849272597394328def7e328"
server_key_iv: "6c596a7d2b7ed2dc5b1d2e627b6aeb93"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.229176 28937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "28911e71554f408e9a0eb561fcd90ec9"
format_stamp: "Formatted at 2026-08-10 21:02:01 on dist-test-slave-1x4g"
server_key: "3a850baae849272597394328def7e328"
server_key_iv: "6c596a7d2b7ed2dc5b1d2e627b6aeb93"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.231256 28937 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 21:02:01.243920 28953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.244122 28937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:01.244227 28937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "28911e71554f408e9a0eb561fcd90ec9"
format_stamp: "Formatted at 2026-08-10 21:02:01 on dist-test-slave-1x4g"
server_key: "3a850baae849272597394328def7e328"
server_key_iv: "6c596a7d2b7ed2dc5b1d2e627b6aeb93"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.244329 28937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:01.282752 28937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:01.285226 28937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:01.285384 28937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:01.285615 28937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:01.285979 28937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:01.286028 28937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.286067 28937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:01.286092 28937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.292550 28937 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:44391
I20260810 21:02:01.293761 28937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:01.294759 29074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:44391 every 8 connection(s)
I20260810 21:02:01.295639 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 28937
I20260810 21:02:01.295750 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:02:01.295943 15136 external_mini_cluster.cc:1428] Setting key 10af2180c2630d0fbd136902f4ddc902
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:01.308054 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.296118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:34923 (local address 127.14.200.62:41369)
0810 21:02:01.296166 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:01.296168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:01.297058 (+ 890us) server_negotiation.cc:407] Connection header received
0810 21:02:01.297779 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:01.297780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:01.297795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:01.297827 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:01.298698 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.299465 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.301116 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.301406 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.301454 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.304209 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:01.304210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:01.304211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:01.304218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:01.305995 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.306481 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.306482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.306483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.306553 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.306833 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.306833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.306834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.306964 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:01.307008 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:01.307878 (+ 870us) server_negotiation.cc:299] Negotiation successful
0810 21:02:01.307964 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:02:01.308604 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.295838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.3:34923)
0810 21:02:01.296935 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:01.296981 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:01.297654 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:01.298031 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:01.298035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:01.298291 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:01.298589 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.298596 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.299638 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.299639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.301014 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.301017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.301573 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.301574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.301713 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.302315 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:01.302327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:01.304099 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:01.306097 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.306103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.306106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.306389 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.306627 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.306628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.306628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.306757 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.307292 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:01.307298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:01.307700 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 21:02:01.308191 (+ 491us) client_negotiation.cc:240] Negotiation successful
0810 21:02:01.308354 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"thread_start_us":80,"threads_started":1}
I20260810 21:02:01.309059 29075 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:01.309156 29075 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.309342 29075 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:01.309762 28567 ts_manager.cc:194] Registered new tserver with Master: 28911e71554f408e9a0eb561fcd90ec9 (127.14.200.3:44391)
I20260810 21:02:01.310225 28567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:34923
I20260810 21:02:01.318367 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:01.319137 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 28651
I20260810 21:02:01.330765 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 28793
I20260810 21:02:01.339753 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 28937
I20260810 21:02:01.348569 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 28532
I20260810 21:02:01.357977 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41369
--webserver_interface=127.14.200.62
--webserver_port=37187
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41369
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:01.441931 29078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:01.442131 29078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:01.442152 29078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:01.443270 29078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:01.443312 29078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:01.443326 29078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:01.443338 29078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:01.443351 29078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:01.445103 29078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41369
--ranger_config_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=37187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:01.445350 29078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:01.445578 29078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:01.450183 29083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.450222 29084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.450420 29086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:01.450724 29078 server_base.cc:1034] running on GCE node
I20260810 21:02:01.450963 29078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:01.451282 29078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:01.452440 29078 hybrid_clock.cc:648] HybridClock initialized: now 1786395721452418 us; error 38 us; skew 500 ppm
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:01.455034 29078 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:01.455499 29078 webserver.cc:460] Webserver started at http://127.14.200.62:37187/ using document root <none> and password file <none>
I20260810 21:02:01.455740 29078 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:01.455816 29078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:01.502758 29078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:01.503544 29095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.503707 29078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:01.503806 29078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "a103532b8da943e8a45d9c1dcc3e48a1"
format_stamp: "Formatted at 2026-08-10 21:01:59 on dist-test-slave-1x4g"
server_key: "071a311c215ce88c009df463c571d2b3"
server_key_iv: "137bdb3650ce64a95daef6be82fe29bd"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.504195 29078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:01.524537 29078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:01.526523 29078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:01.526674 29078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:01.531687 29078 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41369
I20260810 21:02:01.531731 29152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41369 every 8 connection(s)
I20260810 21:02:01.532120 29078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:01.533787 29153 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:02:01.534696 29153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Bootstrap starting.
I20260810 21:02:01.535137 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29078
I20260810 21:02:01.535465 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:42789
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:41369
--webserver_port=36323
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:01.545492 29153 log.cc:824] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:01.548031 29153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:01.548410 29153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Bootstrap complete.
I20260810 21:02:01.552057 29153 raft_consensus.cc:348] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:02:01.552417 29153 raft_consensus.cc:729] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a103532b8da943e8a45d9c1dcc3e48a1, State: Initialized, Role: FOLLOWER
I20260810 21:02:01.552603 29153 consensus_queue.cc:260] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:02:01.552677 29153 raft_consensus.cc:388] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:01.552724 29153 raft_consensus.cc:482] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:01.552764 29153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:02:01.554204 29153 raft_consensus.cc:504] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:02:01.554325 29153 leader_election.cc:302] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a103532b8da943e8a45d9c1dcc3e48a1; no voters:
I20260810 21:02:01.554505 29153 leader_election.cc:288] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 21:02:01.554620 29157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 21:02:01.554795 29157 raft_consensus.cc:686] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [term 2 LEADER]: Becoming Leader. State: Replica: a103532b8da943e8a45d9c1dcc3e48a1, State: Running, Role: LEADER
I20260810 21:02:01.554836 29153 sys_catalog.cc:564] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:01.554983 29157 consensus_queue.cc:237] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } }
I20260810 21:02:01.555316 29157 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } } }
I20260810 21:02:01.555446 29157 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:01.555624 29157 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a103532b8da943e8a45d9c1dcc3e48a1. Latest consensus state: current_term: 2 leader_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103532b8da943e8a45d9c1dcc3e48a1" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41369 } } }
I20260810 21:02:01.555728 29157 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:01.556689 29153 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 21:02:01.559298 29168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:01.560384 29168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:01.560547 29168 catalog_manager.cc:1259] Loaded cluster ID: 450878ac224f4e269a9e8738da98ee58
I20260810 21:02:01.560590 29168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:02:01.562103 29168 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:01.562300 29168 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Loaded TSK: 0
I20260810 21:02:01.562515 29168 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:02:01.646334 29155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:01.646595 29155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:01.646626 29155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:01.648113 29155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:01.648166 29155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:01.648202 29155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:01.650210 29155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:42789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=36323
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:01.650449 29155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:01.650691 29155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:01.657075 29194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.657114 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:01.658237 29155 server_base.cc:1034] running on GCE node
W20260810 21:02:01.658079 29193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:01.658574 29155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:01.658838 29155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:01.660032 29155 hybrid_clock.cc:648] HybridClock initialized: now 1786395721659994 us; error 58 us; skew 500 ppm
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:01.663558 29155 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:01.664175 29155 webserver.cc:460] Webserver started at http://127.14.200.1:36323/ using document root <none> and password file <none>
I20260810 21:02:01.664471 29155 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:01.664568 29155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:01.693274 29155 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260810 21:02:01.694558 29205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.694742 29155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:01.694829 29155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "3298154ecf8b474199c7b400cda2b641"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "3e84f287c486f736d3d07a90c1b01f97"
server_key_iv: "3df04bfd442e1262b6fbf1c70554c4b0"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.695343 29155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:01.737087 29155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:01.739717 29155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:01.739918 29155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:01.740162 29155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:01.740513 29155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:01.740567 29155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.740612 29155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:01.740649 29155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.747084 29155 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42789
I20260810 21:02:01.747674 29155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:01.749028 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29155
I20260810 21:02:01.749428 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:42101
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:41369
--webserver_port=36749
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:01.751971 29326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42789 every 8 connection(s)
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:01.767876 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.752699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.1:55471)
0810 21:02:01.754374 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:01.754427 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:01.755210 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:01.756135 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:01.756140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:01.756404 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:01.756726 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.756734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.758662 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.758664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.759964 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.759968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.761088 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.761090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.761270 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.761945 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:01.761957 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:01.763703 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:01.766049 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.766056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.766058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.766336 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.766616 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.766617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.766617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.766734 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.767204 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:01.767210 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:01.767358 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 21:02:01.767461 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 21:02:01.767620 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":129,"threads_started":1}
I20260810 21:02:01.768692 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.753008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:55471 (local address 127.14.200.62:41369)
0810 21:02:01.755527 (+ 2519us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:01.755537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:01.755582 (+ 45us) server_negotiation.cc:407] Connection header received
0810 21:02:01.755740 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:01.755747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:01.755806 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:01.755963 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:01.756941 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.758499 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.760083 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.760827 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.760957 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.763825 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:01.763830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:01.763830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:01.763842 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:01.765956 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.766422 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.766425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.766426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.766547 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.766799 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.766800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.766800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.767099 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:01.767176 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:01.768338 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0810 21:02:01.768471 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2456,"thread_start_us":83,"threads_started":1}
I20260810 21:02:01.769798 29328 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:01.769908 29328 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.770107 29328 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:01.771104 29328 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:41369 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 21:02:01.771170 29328 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.771234 29328 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:01.771606 29328 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.771677 29328 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:01.772002 29328 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:02:01.860530 29330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:01.860760 29330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:01.860812 29330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:01.862306 29330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:01.862390 29330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:01.862437 29330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:01.864454 29330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:42101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=36749
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:01.864699 29330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:01.865022 29330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:01.870031 29337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.870232 29340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:01.870046 29338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:01.870554 29330 server_base.cc:1034] running on GCE node
I20260810 21:02:01.870762 29330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:01.871029 29330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:01.872193 29330 hybrid_clock.cc:648] HybridClock initialized: now 1786395721872174 us; error 35 us; skew 500 ppm
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:01.874874 29330 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:01.875351 29330 webserver.cc:460] Webserver started at http://127.14.200.2:36749/ using document root <none> and password file <none>
I20260810 21:02:01.875627 29330 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:01.875710 29330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:01.893234 29330 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260810 21:02:01.894299 29348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.894520 29330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:01.894618 29330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "037e57b48a924e11bbc04f4b15ef2b28"
format_stamp: "Formatted at 2026-08-10 21:02:00 on dist-test-slave-1x4g"
server_key: "5174d71a40148a4f0436a4c5ba9af711"
server_key_iv: "f0b9c375ad5610e10561aea9f9b95884"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:01.894991 29330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:01.910285 29330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:01.912583 29330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:01.912767 29330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:01.913097 29330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:01.913483 29330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:01.913569 29330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.913621 29330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:01.913657 29330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:01.921527 29330 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:42101
I20260810 21:02:01.922088 29330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:02:01.922574 29473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:42101 every 8 connection(s)
I20260810 21:02:01.927122 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29330
I20260810 21:02:01.927430 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.200.20:56727/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:44391
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:41369
--webserver_port=38225
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 21:02:01 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395721, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:01.932272 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.922980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:33697 (local address 127.14.200.62:41369)
0810 21:02:01.923047 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:01.923049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:01.923264 (+ 215us) server_negotiation.cc:407] Connection header received
0810 21:02:01.923833 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:01.923835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:01.923849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:01.923896 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:01.924483 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.925085 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.926203 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.926399 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.926488 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.928962 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:01.928964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:01.928964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:01.928972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:01.930708 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.931209 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.931210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.931211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.931265 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:01.931524 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:01.931524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:01.931525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:01.931657 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:01.931696 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:01.932053 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 21:02:01.932139 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:02:01.933112 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:01.922970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.2:33697)
0810 21:02:01.923145 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:01.923180 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:01.923712 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:01.923982 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:01.923988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:01.924169 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:01.924362 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.924367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.925181 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.925182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.926090 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:01.926094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:01.926521 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:01.926523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:01.926637 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:01.927184 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:01.927196 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:01.928846 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:01.930804 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.930810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.930813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.931126 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.931330 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:01.931331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:01.931331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:01.931455 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:01.931740 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:01.931745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:01.931898 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 21:02:01.932612 (+ 714us) client_negotiation.cc:240] Negotiation successful
0810 21:02:01.932823 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260810 21:02:01.934015 29475 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:01.934144 29475 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.934334 29475 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:01.935057 29475 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:41369 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 21:02:01.935140 29475 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.935235 29475 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:01.935592 29475 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:01.935674 29475 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:01.936053 29475 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:02:02.031814 29478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:02.032150 29478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:02.032200 29478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:02.033937 29478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:02.033995 29478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:02.034034 29478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:02.036062 29478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.200.20:56727/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:44391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=38225
--tserver_master_addrs=127.14.200.62:41369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:02.036281 29478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:02.036562 29478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:02.043036 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:02.043049 29483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:02.043291 29478 server_base.cc:1034] running on GCE node
W20260810 21:02:02.044942 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:02.045181 29478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:02.045449 29478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:02.046628 29478 hybrid_clock.cc:648] HybridClock initialized: now 1786395722046600 us; error 43 us; skew 500 ppm
Aug 10 21:02:02 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:02.049824 29478 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:02.050391 29478 webserver.cc:460] Webserver started at http://127.14.200.3:38225/ using document root <none> and password file <none>
I20260810 21:02:02.050679 29478 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:02.050796 29478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 21:02:02 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:02.084942 29478 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260810 21:02:02.087332 29496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:02.087533 29478 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 21:02:02.087639 29478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "28911e71554f408e9a0eb561fcd90ec9"
format_stamp: "Formatted at 2026-08-10 21:02:01 on dist-test-slave-1x4g"
server_key: "3a850baae849272597394328def7e328"
server_key_iv: "6c596a7d2b7ed2dc5b1d2e627b6aeb93"
server_key_version: "kuduclusterkey@0"
I20260810 21:02:02.088156 29478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:02.113952 29478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:02.116462 29478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:02.116657 29478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:02.116961 29478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:02.117385 29478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:02.117440 29478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:02.117472 29478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:02.117547 29478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:02.125283 29478 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:44391
I20260810 21:02:02.125689 29478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:02.126313 29615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:44391 every 8 connection(s)
I20260810 21:02:02.126494 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29478
Aug 10 21:02:02 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:02.136536 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:02.126800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:48725 (local address 127.14.200.62:41369)
0810 21:02:02.126854 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:02.126857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:02.127014 (+ 157us) server_negotiation.cc:407] Connection header received
0810 21:02:02.127587 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:02.127589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:02.127602 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:02.127683 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:02.128262 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.129057 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.130091 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.130299 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.130375 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:02.132962 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:02.132964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:02.132964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:02.132972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:02.134858 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:02.135451 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:02.135452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:02.135452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:02.135506 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:02.135778 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:02.135779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:02.135779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:02.135913 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:02.136008 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:02.136330 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 21:02:02.136439 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:02:02.137264 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:02.126489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41369 (local address 127.14.200.3:48725)
0810 21:02:02.126831 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:02.126929 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:02.127499 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:02.127781 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:02.127786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:02.127977 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:02.128167 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:02.128172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.129149 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.129150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:02.129998 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:02.130001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.130408 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.130409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:02.130533 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:02.131104 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:02.131114 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:02.132849 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:02.134948 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:02.134955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:02.134957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:02.135181 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:02.135590 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:02.135591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:02.135591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:02.135708 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:02.135979 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:02.135985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:02.136151 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 21:02:02.136909 (+ 758us) client_negotiation.cc:240] Negotiation successful
0810 21:02:02.137072 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":80,"threads_started":1}
I20260810 21:02:02.137766 29616 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41369
I20260810 21:02:02.137890 29616 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:02.138079 29616 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:02.138727 29616 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:41369 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 21:02:02.138806 29616 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:02.138870 29616 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:02.139200 29616 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:02.139269 29616 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
W20260810 21:02:02.139536 29616 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 21:02:02.140729 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:02.126890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.14.200.62:41369)
0810 21:02:02.127644 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:02.127646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:02.127653 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:02.127700 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:02.127700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:02.127713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:02.127771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:02.129045 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.130297 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.131742 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:02.131952 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:02.132045 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:02.134372 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:02.134373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:02.134374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:02.134381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:02.136075 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:02.139933 (+ 3858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:02.139935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:02.139935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:02.139989 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:02.140212 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:02.140212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:02.140214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:02.140310 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:02.140501 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:02.140545 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 21:02:02.140624 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":95,"threads_started":1}
Aug 10 21:02:02 dist-test-slave-1x4g krb5kdc[28055](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:02.772203 29328 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:02.772354 29328 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:02.772810 29114 ts_manager.cc:194] Registered new tserver with Master: 3298154ecf8b474199c7b400cda2b641 (127.14.200.1:42789)
I20260810 21:02:02.773846 29114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:55471
I20260810 21:02:02.936229 29475 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:02.936368 29475 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:02.936812 29114 ts_manager.cc:194] Registered new tserver with Master: 037e57b48a924e11bbc04f4b15ef2b28 (127.14.200.2:42101)
I20260810 21:02:02.937350 29114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:33697
Aug 10 21:02:02 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
I20260810 21:02:03.139720 29616 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:03.139865 29616 heartbeater.cc:511] Master 127.14.200.62:41369 requested a full tablet report, sending...
I20260810 21:02:03.140223 29114 ts_manager.cc:194] Registered new tserver with Master: 28911e71554f408e9a0eb561fcd90ec9 (127.14.200.3:44391)
I20260810 21:02:03.140717 29114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:48725
I20260810 21:02:03.142268 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:03.153771 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.143221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.14.200.62:41369)
0810 21:02:03.143552 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.143555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.143778 (+ 223us) server_negotiation.cc:407] Connection header received
0810 21:02:03.143801 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.143802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.143814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.143867 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:03.144586 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.145414 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.146739 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.147145 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.147233 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.149487 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:03.149489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:03.149489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:03.149507 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:03.151426 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:03.152067 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:03.152068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:03.152069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:03.152153 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:03.152489 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:03.152490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:03.152490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:03.152701 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:03.152873 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.153581 (+ 708us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.153669 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":63,"threads_started":1}
I20260810 21:02:03.156229 29114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:02:03.158449 29114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:02:03.174919 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.167473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:42101 (local address 127.0.0.1:44918)
0810 21:02:03.169646 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.169687 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:03.169756 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:03.170451 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:03.170455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:03.170457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:03.170523 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.170525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.171809 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.171811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.173450 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.173453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.174602 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.174603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.174690 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.174704 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 21:02:03.174790 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 21:02:03.174838 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2139,"thread_start_us":73,"threads_started":1}
I20260810 21:02:03.175954 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.167725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.14.200.2:42101)
0810 21:02:03.169890 (+ 2165us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.169896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.169906 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:03.169942 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.169945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.169961 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.170340 (+ 379us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:03.170608 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.171709 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.173551 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.174349 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.174440 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.174504 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.175803 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.175845 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2130,"thread_start_us":56,"threads_started":1}
I20260810 21:02:03.176921 29394 tablet_service.cc:1467] Processing CreateTablet for tablet 853e989f377147fba5bddd4d57d9fb66 (DEFAULT_TABLE table=test-table [id=2ce86e38d9544794959c36da7ef154d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:03.177364 29394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853e989f377147fba5bddd4d57d9fb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:03.181192 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.168091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:44391 (local address 127.0.0.1:53250)
0810 21:02:03.175611 (+ 7520us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.175619 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:03.175652 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:03.176770 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:03.176771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:03.176793 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:03.176825 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.176826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.178057 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.178058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.179036 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.179040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.180968 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.180969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.181028 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.181032 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:03.181084 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 21:02:03.181120 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7511,"thread_start_us":51,"threads_started":1}
I20260810 21:02:03.181608 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.168301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.14.200.3:44391)
0810 21:02:03.173626 (+ 5325us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.173630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.176502 (+ 2872us) server_negotiation.cc:407] Connection header received
0810 21:02:03.176547 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.176549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.176565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.176669 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:03.176927 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.177933 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.180091 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.180801 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.180883 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.180932 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.181480 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.181524 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5289,"thread_start_us":60,"threads_started":1}
I20260810 21:02:03.184162 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.167926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42789 (local address 127.0.0.1:34466)
0810 21:02:03.173692 (+ 5766us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.173704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:03.173777 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:03.174106 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:03.174108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:03.174110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:03.174170 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.174172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.175381 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.175383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.176393 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.176397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.183912 (+ 7515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.183913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.183992 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.183996 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:03.184053 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:02:03.184093 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5747,"thread_start_us":64,"threads_started":1}
I20260810 21:02:03.184365 29547 tablet_service.cc:1467] Processing CreateTablet for tablet 853e989f377147fba5bddd4d57d9fb66 (DEFAULT_TABLE table=test-table [id=2ce86e38d9544794959c36da7ef154d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:03.184536 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.168058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.14.200.1:42789)
0810 21:02:03.173522 (+ 5464us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.173527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.173865 (+ 338us) server_negotiation.cc:407] Connection header received
0810 21:02:03.173912 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.173914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.173931 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.174008 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:03.174258 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.175262 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.179167 (+ 3905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.179904 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.179982 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.180052 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.184394 (+ 4342us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.184455 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5421,"thread_start_us":78,"threads_started":1}
I20260810 21:02:03.184851 29547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853e989f377147fba5bddd4d57d9fb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:03.185340 29250 tablet_service.cc:1467] Processing CreateTablet for tablet 853e989f377147fba5bddd4d57d9fb66 (DEFAULT_TABLE table=test-table [id=2ce86e38d9544794959c36da7ef154d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:03.185734 29250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 853e989f377147fba5bddd4d57d9fb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:03.190762 29655 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Bootstrap starting.
I20260810 21:02:03.191762 29655 tablet_bootstrap.cc:654] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:03.192515 29655 log.cc:824] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:03.192723 29656 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Bootstrap starting.
I20260810 21:02:03.193542 29656 tablet_bootstrap.cc:654] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:03.194741 29656 log.cc:824] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:03.195766 29655 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: No bootstrap required, opened a new log
I20260810 21:02:03.195878 29655 ts_tablet_manager.cc:1397] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 21:02:03.199909 29655 raft_consensus.cc:348] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.200222 29655 raft_consensus.cc:374] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:03.200307 29655 raft_consensus.cc:729] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28911e71554f408e9a0eb561fcd90ec9, State: Initialized, Role: FOLLOWER
I20260810 21:02:03.200513 29655 consensus_queue.cc:260] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.200978 29654 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Bootstrap starting.
I20260810 21:02:03.201046 29655 ts_tablet_manager.cc:1428] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 21:02:03.201202 29616 heartbeater.cc:503] Master 127.14.200.62:41369 was elected leader, sending a full tablet report...
I20260810 21:02:03.202224 29654 tablet_bootstrap.cc:654] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:03.203004 29654 log.cc:824] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:03.203328 29656 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: No bootstrap required, opened a new log
I20260810 21:02:03.203464 29656 ts_tablet_manager.cc:1397] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260810 21:02:03.207888 29654 tablet_bootstrap.cc:492] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: No bootstrap required, opened a new log
I20260810 21:02:03.208066 29654 ts_tablet_manager.cc:1397] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260810 21:02:03.208734 29656 raft_consensus.cc:348] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.209052 29656 raft_consensus.cc:374] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:03.209168 29656 raft_consensus.cc:729] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3298154ecf8b474199c7b400cda2b641, State: Initialized, Role: FOLLOWER
I20260810 21:02:03.209421 29656 consensus_queue.cc:260] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.209844 29656 ts_tablet_manager.cc:1428] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 21:02:03.210090 29328 heartbeater.cc:503] Master 127.14.200.62:41369 was elected leader, sending a full tablet report...
I20260810 21:02:03.212069 29654 raft_consensus.cc:348] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.212277 29654 raft_consensus.cc:374] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:03.212365 29654 raft_consensus.cc:729] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037e57b48a924e11bbc04f4b15ef2b28, State: Initialized, Role: FOLLOWER
I20260810 21:02:03.212559 29654 consensus_queue.cc:260] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.212939 29654 ts_tablet_manager.cc:1428] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260810 21:02:03.213090 29475 heartbeater.cc:503] Master 127.14.200.62:41369 was elected leader, sending a full tablet report...
W20260810 21:02:03.254215 29329 tablet.cc:2369] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 21:02:03 dist-test-slave-1x4g krb5kdc[28055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395722, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for HTTP/127.14.200.20@KRBTEST.COM
W20260810 21:02:03.377113 29617 tablet.cc:2369] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:03.407199 29659 raft_consensus.cc:482] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:03.407404 29659 raft_consensus.cc:504] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.407873 29659 leader_election.cc:288] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3298154ecf8b474199c7b400cda2b641 (127.14.200.1:42789), 037e57b48a924e11bbc04f4b15ef2b28 (127.14.200.2:42101)
I20260810 21:02:03.418009 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.408789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:44721 (local address 127.14.200.2:42101)
0810 21:02:03.408854 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.408856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.411012 (+ 2156us) server_negotiation.cc:407] Connection header received
0810 21:02:03.411131 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.411132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.411142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.411172 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:03.411413 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.412154 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.417029 (+ 4875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.417615 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.417662 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.417672 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.417894 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.417935 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:02:03.418521 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.408290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:42101 (local address 127.14.200.3:44721)
0810 21:02:03.410928 (+ 2638us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.410942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:03.411046 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:03.411246 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:03.411247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:03.411249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:03.411333 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.411336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.412398 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.412399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.413711 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.413715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.417749 (+ 4034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.417750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.417820 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.417825 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:02:03.418359 (+ 534us) client_negotiation.cc:240] Negotiation successful
0810 21:02:03.418400 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2616,"thread_start_us":112,"threads_started":1}
I20260810 21:02:03.419060 29421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853e989f377147fba5bddd4d57d9fb66" candidate_uuid: "28911e71554f408e9a0eb561fcd90ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037e57b48a924e11bbc04f4b15ef2b28" is_pre_election: true
I20260810 21:02:03.419251 29421 raft_consensus.cc:2454] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28911e71554f408e9a0eb561fcd90ec9 in term 0.
I20260810 21:02:03.419612 29497 leader_election.cc:302] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 037e57b48a924e11bbc04f4b15ef2b28, 28911e71554f408e9a0eb561fcd90ec9; no voters:
I20260810 21:02:03.419744 29659 raft_consensus.cc:2781] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:03.419780 29659 raft_consensus.cc:482] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:03.419800 29659 raft_consensus.cc:3037] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:03.421087 29659 raft_consensus.cc:504] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.421146 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.409015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38935 (local address 127.14.200.1:42789)
0810 21:02:03.409067 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.409069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.417136 (+ 8067us) server_negotiation.cc:407] Connection header received
0810 21:02:03.417239 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.417240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.417251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.417280 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:03.417467 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.418117 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.420133 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.420723 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.420793 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.420803 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.421029 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.421078 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:02:03.421240 29659 leader_election.cc:288] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [CANDIDATE]: Term 1 election: Requested vote from peers 3298154ecf8b474199c7b400cda2b641 (127.14.200.1:42789), 037e57b48a924e11bbc04f4b15ef2b28 (127.14.200.2:42101)
I20260810 21:02:03.421386 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.408942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42789 (local address 127.14.200.3:38935)
0810 21:02:03.417067 (+ 8125us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.417079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:03.417166 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:03.417340 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:03.417341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:03.417342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:03.417400 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.417401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.418225 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.418225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.420027 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:03.420029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.420864 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.420864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:03.420943 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.420947 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:03.421250 (+ 303us) client_negotiation.cc:240] Negotiation successful
0810 21:02:03.421294 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8102,"thread_start_us":93,"threads_started":1}
I20260810 21:02:03.421744 29276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853e989f377147fba5bddd4d57d9fb66" candidate_uuid: "28911e71554f408e9a0eb561fcd90ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3298154ecf8b474199c7b400cda2b641" is_pre_election: true
I20260810 21:02:03.421846 29421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853e989f377147fba5bddd4d57d9fb66" candidate_uuid: "28911e71554f408e9a0eb561fcd90ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "037e57b48a924e11bbc04f4b15ef2b28"
I20260810 21:02:03.421909 29276 raft_consensus.cc:2454] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28911e71554f408e9a0eb561fcd90ec9 in term 0.
I20260810 21:02:03.421938 29421 raft_consensus.cc:3037] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:03.422086 29275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "853e989f377147fba5bddd4d57d9fb66" candidate_uuid: "28911e71554f408e9a0eb561fcd90ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3298154ecf8b474199c7b400cda2b641"
I20260810 21:02:03.422179 29275 raft_consensus.cc:3037] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:03.423175 29275 raft_consensus.cc:2454] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28911e71554f408e9a0eb561fcd90ec9 in term 1.
I20260810 21:02:03.423302 29421 raft_consensus.cc:2454] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28911e71554f408e9a0eb561fcd90ec9 in term 1.
I20260810 21:02:03.423591 29500 leader_election.cc:302] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28911e71554f408e9a0eb561fcd90ec9, 3298154ecf8b474199c7b400cda2b641; no voters:
W20260810 21:02:03.423609 29476 tablet.cc:2369] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:03.423967 29659 raft_consensus.cc:2781] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:03.424202 29659 raft_consensus.cc:686] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 1 LEADER]: Becoming Leader. State: Replica: 28911e71554f408e9a0eb561fcd90ec9, State: Running, Role: LEADER
I20260810 21:02:03.424297 29659 consensus_queue.cc:237] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } }
I20260810 21:02:03.425199 29114 catalog_manager.cc:5495] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28911e71554f408e9a0eb561fcd90ec9 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "28911e71554f408e9a0eb561fcd90ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28911e71554f408e9a0eb561fcd90ec9" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 44391 } health_report { overall_health: HEALTHY } } }
I20260810 21:02:03.455825 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.452165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.14.200.3:44391)
0810 21:02:03.452213 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:03.452215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:03.452223 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:02:03.452267 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:03.452267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:03.452283 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:03.452366 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:03.452512 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.453209 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.454361 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:03.454695 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:03.454753 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:03.454989 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:03.455131 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:03.455220 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:03.455723 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 21:02:03.455746 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:02:03.458299 29421 raft_consensus.cc:1261] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 1 FOLLOWER]: Refusing update from remote peer 28911e71554f408e9a0eb561fcd90ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:03.458770 29667 consensus_queue.cc:1035] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "037e57b48a924e11bbc04f4b15ef2b28" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 42101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:03.461233 29275 raft_consensus.cc:1261] T 853e989f377147fba5bddd4d57d9fb66 P 3298154ecf8b474199c7b400cda2b641 [term 1 FOLLOWER]: Refusing update from remote peer 28911e71554f408e9a0eb561fcd90ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:03.461588 29667 consensus_queue.cc:1035] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3298154ecf8b474199c7b400cda2b641" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:03.464422 29670 mvcc.cc:205] Tried to move back new op lower bound from 7317076883282010112 to 7317076883146461184. Current Snapshot: MvccSnapshot[applied={T|T < 7317076883282010112}]
I20260810 21:02:03.465272 29671 mvcc.cc:205] Tried to move back new op lower bound from 7317076883282010112 to 7317076883146461184. Current Snapshot: MvccSnapshot[applied={T|T < 7317076883282010112}]
I20260810 21:02:03.467020 29113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:03.467113 29113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:03.467969 29113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a103532b8da943e8a45d9c1dcc3e48a1: Sending DeleteTablet for 3 replicas of tablet 853e989f377147fba5bddd4d57d9fb66
I20260810 21:02:03.468331 29394 tablet_service.cc:1514] Processing DeleteTablet for tablet 853e989f377147fba5bddd4d57d9fb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:03 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:44918
I20260810 21:02:03.468487 29250 tablet_service.cc:1514] Processing DeleteTablet for tablet 853e989f377147fba5bddd4d57d9fb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:03 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:34466
I20260810 21:02:03.468673 29547 tablet_service.cc:1514] Processing DeleteTablet for tablet 853e989f377147fba5bddd4d57d9fb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:03 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:53250
I20260810 21:02:03.469472 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29155
I20260810 21:02:03.473675 29678 tablet_replica.cc:330] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: stopping tablet replica
I20260810 21:02:03.473855 29678 raft_consensus.cc:2229] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:03.474018 29678 raft_consensus.cc:2258] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:03.476917 29680 tablet_replica.cc:330] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: stopping tablet replica
I20260810 21:02:03.477255 29680 raft_consensus.cc:2229] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:02:03.478288 29680 raft_consensus.cc:2258] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:03.478955 29680 ts_tablet_manager.cc:1905] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:03.479024 29678 ts_tablet_manager.cc:1905] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:03.480852 29680 ts_tablet_manager.cc:1918] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:03.480974 29680 log.cc:1196] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/853e989f377147fba5bddd4d57d9fb66
I20260810 21:02:03.480999 29678 ts_tablet_manager.cc:1918] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:03.481178 29678 log.cc:1196] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/853e989f377147fba5bddd4d57d9fb66
I20260810 21:02:03.481287 29680 ts_tablet_manager.cc:1939] T 853e989f377147fba5bddd4d57d9fb66 P 28911e71554f408e9a0eb561fcd90ec9: Deleting consensus metadata
I20260810 21:02:03.481560 29678 ts_tablet_manager.cc:1939] T 853e989f377147fba5bddd4d57d9fb66 P 037e57b48a924e11bbc04f4b15ef2b28: Deleting consensus metadata
I20260810 21:02:03.481856 29101 catalog_manager.cc:4841] TS 28911e71554f408e9a0eb561fcd90ec9 (127.14.200.3:44391): tablet 853e989f377147fba5bddd4d57d9fb66 (table test-table [id=2ce86e38d9544794959c36da7ef154d5]) successfully deleted
I20260810 21:02:03.485307 29097 catalog_manager.cc:4841] TS 037e57b48a924e11bbc04f4b15ef2b28 (127.14.200.2:42101): tablet 853e989f377147fba5bddd4d57d9fb66 (table test-table [id=2ce86e38d9544794959c36da7ef154d5]) successfully deleted
W20260810 21:02:03.486408 29500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:42789 (error 108)
W20260810 21:02:03.486470 29101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:42789 (error 108)
I20260810 21:02:03.487360 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.487198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42789 (local address 127.0.0.1:34478)
0810 21:02:03.487268 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.487325 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 21:02:03.487358 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29330
W20260810 21:02:03.487535 29101 catalog_manager.cc:4584] TS 3298154ecf8b474199c7b400cda2b641 (127.14.200.1:42789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 853e989f377147fba5bddd4d57d9fb66: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42789: connect: Connection refused (error 111)
I20260810 21:02:03.491091 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:03.490981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42789 (local address 127.14.200.3:60897)
0810 21:02:03.491026 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:03.491061 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 21:02:03.497495 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29478
I20260810 21:02:03.514678 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29078
I20260810 21:02:03.524248 15136 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 21:02:03.868952 15136 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 21:02:03.869047 15136 mini_ranger.cc:66] Stopping Ranger...
I20260810 21:02:04.214531 15136 mini_ranger.cc:68] Stopped Ranger
2026-08-10 21:02:04.214 UTC [28086] LOG: received smart shutdown request
2026-08-10 21:02:04.218 UTC [28086] LOG: background worker "logical replication launcher" (PID 28096) exited with exit code 1
2026-08-10 21:02:04.219 UTC [28090] LOG: shutting down
2026-08-10 21:02:04.367 UTC [28086] LOG: database system is shut down
2026-08-10T21:02:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46156 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:02:04 dist-test-slave-1x4g krb5kdc[29692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:02:04 dist-test-slave-1x4g krb5kdc[29692](info): set up 2 sockets
Aug 10 21:02:04 dist-test-slave-1x4g krb5kdc[29692](info): commencing operation
krb5kdc: starting...
W20260810 21:02:05.547343 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:02:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:02:05.667722 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42749
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:05.747419 29705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:05.747605 29705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:05.747625 29705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:05.748574 29705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:05.748605 29705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:05.748618 29705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:05.748629 29705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:05.748641 29705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:05.749938 29705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:42749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:05.750121 29705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:05.750295 29705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:05.754936 29714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:05.754972 29711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:05.754987 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:05.755025 29705 server_base.cc:1034] running on GCE node
I20260810 21:02:05.755378 29705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:05.755657 29705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:05.756842 29705 hybrid_clock.cc:648] HybridClock initialized: now 1786395725756821 us; error 34 us; skew 500 ppm
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:05.762399 29705 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:05.762872 29705 webserver.cc:460] Webserver started at http://127.14.200.62:45095/ using document root <none> and password file <none>
I20260810 21:02:05.763096 29705 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:05.763162 29705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:05.763311 29705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:05.764390 29705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "82afff2b34c34efa99d31d9e5737be75"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "73a1a22dd0761b4ea8cb16225a0e3866"
server_key_iv: "f83f29425fd1ba220ea6736117da4afc"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.764763 29705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "82afff2b34c34efa99d31d9e5737be75"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "73a1a22dd0761b4ea8cb16225a0e3866"
server_key_iv: "f83f29425fd1ba220ea6736117da4afc"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.766106 29705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:05.766793 29721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:05.766958 29705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:05.767078 29705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "82afff2b34c34efa99d31d9e5737be75"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "73a1a22dd0761b4ea8cb16225a0e3866"
server_key_iv: "f83f29425fd1ba220ea6736117da4afc"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.767149 29705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:05.797035 29705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:05.798909 29705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:05.799090 29705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:05.803280 29705 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:42749
I20260810 21:02:05.803339 29776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:42749 every 8 connection(s)
I20260810 21:02:05.803690 29705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:05.804575 29777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:05.806171 29777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:05.812618 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29705
I20260810 21:02:05.812740 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:02:05.812958 15136 external_mini_cluster.cc:1428] Setting key 598b8807fa5c316482e13c087024124c
I20260810 21:02:05.815251 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.806806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42749 (local address 127.0.0.1:34188)
0810 21:02:05.806978 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.807024 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:05.807452 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:05.807702 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:05.807704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:05.807946 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:05.808098 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.808102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.809165 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.809166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.809823 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.809825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.810279 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.810280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.810337 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.810745 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:05.810753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:05.812028 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:05.813789 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.813796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.813798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.813989 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.814237 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.814238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.814238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.814328 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.814687 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:05.814692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:05.814759 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 21:02:05.814855 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 21:02:05.815081 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":96,"thread_start_us":65,"threads_started":1}
I20260810 21:02:05.815256 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.806851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.14.200.62:42749)
0810 21:02:05.807037 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:05.807041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:05.807105 (+ 64us) server_negotiation.cc:407] Connection header received
0810 21:02:05.807582 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:05.807584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:05.807600 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:05.807672 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:05.808178 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.809086 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.809897 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.810206 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.810280 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.812104 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:05.812111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:05.812111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:05.812117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:05.813688 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.814080 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.814083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.814084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.814163 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.814406 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.814407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.814407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.814612 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:05.814704 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:05.814882 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 21:02:05.815017 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260810 21:02:05.815915 29777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:02:05.816327 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.816172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48142)
0810 21:02:05.816242 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.816294 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 21:02:05.816473 29725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
I20260810 21:02:05.816907 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.816832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48152)
0810 21:02:05.816861 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.816885 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260810 21:02:05.817039 29777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:38703: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.200.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
I20260810 21:02:05.837121 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:38703
--webserver_interface=127.14.200.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:05.866238 29777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } attempt: 1
I20260810 21:02:05.866677 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.866553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48164)
0810 21:02:05.866614 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.866650 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 21:02:05.867086 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.867002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48180)
0810 21:02:05.867045 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.867070 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:02:05.867205 29777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:38703: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
W20260810 21:02:05.914687 29784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:05.914872 29784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:05.914893 29784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:05.915988 29784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:05.916033 29784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:05.916049 29784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:05.916085 29784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:05.916113 29784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:05.917644 29784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.200.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:05.917891 29784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:05.918099 29784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:05.923014 29793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:05.923015 29790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:05.923050 29791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:05.923228 29784 server_base.cc:1034] running on GCE node
I20260810 21:02:05.923504 29784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:05.923785 29784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:05.924971 29784 hybrid_clock.cc:648] HybridClock initialized: now 1786395725924958 us; error 43 us; skew 500 ppm
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:05.927389 29784 init.cc:377] Logged in from keytab as kudu/127.14.200.61@KRBTEST.COM (short username kudu)
I20260810 21:02:05.927757 29784 webserver.cc:460] Webserver started at http://127.14.200.61:44185/ using document root <none> and password file <none>
I20260810 21:02:05.927951 29784 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:05.927989 29784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:05.928073 29784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:05.929072 29784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data/instance:
uuid: "c35ed759a15d42c4b72a163125c64be8"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "68c8d0a579057f35de72b60889d02e11"
server_key_iv: "59c4d0c07ec368ba45940580cf69bbf0"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.929394 29784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal/instance:
uuid: "c35ed759a15d42c4b72a163125c64be8"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "68c8d0a579057f35de72b60889d02e11"
server_key_iv: "59c4d0c07ec368ba45940580cf69bbf0"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.930630 29784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:02:05.931309 29801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:05.931526 29784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:05.931638 29784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "c35ed759a15d42c4b72a163125c64be8"
format_stamp: "Formatted at 2026-08-10 21:02:05 on dist-test-slave-1x4g"
server_key: "68c8d0a579057f35de72b60889d02e11"
server_key_iv: "59c4d0c07ec368ba45940580cf69bbf0"
server_key_version: "encryptionkey@0"
I20260810 21:02:05.931732 29784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:05.935375 29777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } attempt: 2
I20260810 21:02:05.936022 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.935663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48186)
0810 21:02:05.935853 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.935952 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 21:02:05.936434 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.936329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48192)
0810 21:02:05.936377 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.936401 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:02:05.939814 29777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:38703: Network error: Client connection negotiation failed: client connection to 127.14.200.61:38703: connect: Connection refused (error 111)
I20260810 21:02:05.951923 29784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:05.953840 29784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:05.953994 29784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:05.958014 29784 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.61:38703
I20260810 21:02:05.958047 29856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.61:38703 every 8 connection(s)
I20260810 21:02:05.958384 29784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
I20260810 21:02:05.959244 29857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:05.960851 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:02:05.961930 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29784
I20260810 21:02:05.962037 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-1/wal/instance
I20260810 21:02:05.962232 15136 external_mini_cluster.cc:1428] Setting key 42e2fa8f532f551ff4589c22a3fa043b
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:05.970542 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.961521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.14.200.62:42749)
0810 21:02:05.961578 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:05.961592 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:05.961847 (+ 255us) server_negotiation.cc:407] Connection header received
0810 21:02:05.962501 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:05.962502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:05.962514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:05.962570 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:05.963390 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.963896 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.964929 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.965160 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.965252 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.967447 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:05.967448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:05.967449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:05.967457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:05.969182 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.969569 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.969570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.969571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.969636 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.969872 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.969873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.969873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.970016 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:05.970085 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:05.970338 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 21:02:05.970423 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 21:02:05.970619 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.961495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42749 (local address 127.0.0.1:34192)
0810 21:02:05.961694 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.961733 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:05.962386 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:05.962636 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:05.962640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:05.963004 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:05.963310 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.963316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.964000 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.964001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.964837 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.964844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.965265 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.965266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.965413 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.965854 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:05.965864 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:05.967350 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:05.969259 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.969266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.969268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.969491 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.969705 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.969706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.969706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.969802 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.970089 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:05.970093 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:05.970200 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 21:02:05.970316 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 21:02:05.970444 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260810 21:02:05.971235 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.200.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.60@KRBTEST.COM" created.
Aug 10 21:02:05 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:02:05.978469 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.971476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48206)
0810 21:02:05.971518 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.971528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:05.971568 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:05.971819 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:05.971820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:05.971968 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:05.972041 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.972043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.972900 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.972901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.973359 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:05.973360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.973836 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.973837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:05.973917 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.974209 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:05.974213 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:05.975730 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:05.977512 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.977513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.977513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.977670 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.977889 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:05.977889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:05.977890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:05.977944 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:05.978198 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:05.978199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:05.978233 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 21:02:05.978291 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 21:02:05.978377 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 21:02:05.978662 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.971551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.14.200.61:38703)
0810 21:02:05.971676 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:05.971680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:05.971699 (+ 19us) server_negotiation.cc:407] Connection header received
0810 21:02:05.971720 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:05.971721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:05.971730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:05.971783 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:05.972119 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.972739 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.973428 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:05.973748 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:05.973837 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:05.975822 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:05.975829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:05.975829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:05.975835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:05.977375 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.977760 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.977763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.977763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.977815 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:05.978014 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:05.978015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:05.978015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:05.978135 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:05.978216 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:05.978316 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 21:02:05.978450 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260810 21:02:05.979210 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:02:05.979558 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.979409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54146)
0810 21:02:05.979463 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.979524 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 21:02:05.979703 29804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
I20260810 21:02:05.980140 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:05.980026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54148)
0810 21:02:05.980081 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:05.980110 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 21:02:05.980257 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37391: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
I20260810 21:02:05.989822 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:37391
--webserver_interface=127.14.200.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:06.029476 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } attempt: 1
I20260810 21:02:06.029964 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.029814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54162)
0810 21:02:06.029881 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.029928 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 21:02:06.030284 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.030195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54164)
0810 21:02:06.030237 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.030263 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 21:02:06.030400 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37391: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
I20260810 21:02:06.030964 29777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } attempt: 3
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:02:06.038214 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.031191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48220)
0810 21:02:06.031250 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.031261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.031327 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.031506 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.031507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.031644 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.031706 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.031708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.032593 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.032593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.033118 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.033121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.033651 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.033653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.033719 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.034190 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.034194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.035590 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.037323 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.037324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.037324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.037490 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.037691 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.037691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.037691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.037743 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.038006 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.038007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.038042 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.038088 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.038156 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 21:02:06.038252 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.031281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.14.200.61:38703)
0810 21:02:06.031355 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.031357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.031367 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:06.031396 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.031397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.031408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.031491 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.031774 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.032495 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.033197 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.033583 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.033653 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.035662 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.035664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.035664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.035671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.037247 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.037561 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.037563 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.037563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.037626 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.037816 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.037816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.037817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.037949 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.038011 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.038108 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.038178 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 21:02:06.038574 29777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:02:06.038894 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.038789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54166)
0810 21:02:06.038828 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.038855 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 21:02:06.039214 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.039143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54182)
0810 21:02:06.039179 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.039201 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 21:02:06.039316 29777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37391: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
I20260810 21:02:06.070467 29777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } attempt: 1
I20260810 21:02:06.070896 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.070772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54190)
0810 21:02:06.070830 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.070867 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 21:02:06.070686 29864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:06.070953 29864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:06.070988 29864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 21:02:06.071262 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.071170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54194)
0810 21:02:06.071214 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.071240 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:02:06.071389 29777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37391: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
W20260810 21:02:06.072225 29864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:06.072286 29864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:06.072310 29864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:06.072335 29864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:06.072360 29864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:06.073904 29864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:42749,127.14.200.61:38703,127.14.200.60:37391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:37391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.200.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:06.074177 29864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:06.074399 29864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:06.079041 29870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:06.079067 29872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:06.079125 29864 server_base.cc:1034] running on GCE node
W20260810 21:02:06.079039 29869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:06.079461 29864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:06.079754 29864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:06.080914 29864 hybrid_clock.cc:648] HybridClock initialized: now 1786395726080901 us; error 45 us; skew 500 ppm
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395726, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:06.083218 29864 init.cc:377] Logged in from keytab as kudu/127.14.200.60@KRBTEST.COM (short username kudu)
I20260810 21:02:06.083678 29864 webserver.cc:460] Webserver started at http://127.14.200.60:42235/ using document root <none> and password file <none>
I20260810 21:02:06.083932 29864 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:06.084002 29864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:06.084165 29864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:06.085258 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data/instance:
uuid: "cdd6eafc634644eb9c822f0754749bee"
format_stamp: "Formatted at 2026-08-10 21:02:06 on dist-test-slave-1x4g"
server_key: "271ea8fa8736482c8425b57fb32f6e14"
server_key_iv: "a738d13938c2fad53d4d7494ec11cd97"
server_key_version: "encryptionkey@0"
I20260810 21:02:06.085631 29864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal/instance:
uuid: "cdd6eafc634644eb9c822f0754749bee"
format_stamp: "Formatted at 2026-08-10 21:02:06 on dist-test-slave-1x4g"
server_key: "271ea8fa8736482c8425b57fb32f6e14"
server_key_iv: "a738d13938c2fad53d4d7494ec11cd97"
server_key_version: "encryptionkey@0"
I20260810 21:02:06.086877 29864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:02:06.087597 29879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:06.087787 29864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:02:06.087895 29864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "cdd6eafc634644eb9c822f0754749bee"
format_stamp: "Formatted at 2026-08-10 21:02:06 on dist-test-slave-1x4g"
server_key: "271ea8fa8736482c8425b57fb32f6e14"
server_key_iv: "a738d13938c2fad53d4d7494ec11cd97"
server_key_version: "encryptionkey@0"
I20260810 21:02:06.087970 29864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:06.096576 29864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:06.098315 29864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:06.098466 29864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:06.098567 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } attempt: 2
I20260810 21:02:06.099013 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.098852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54208)
0810 21:02:06.098923 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.098969 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 21:02:06.099400 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.099272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54216)
0810 21:02:06.099326 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.099366 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 21:02:06.099520 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37391: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37391: connect: Connection refused (error 111)
I20260810 21:02:06.102877 29864 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.60:37391
I20260810 21:02:06.102926 29938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.60:37391 every 8 connection(s)
I20260810 21:02:06.103258 29864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
I20260810 21:02:06.104080 29939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:06.104465 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 29864
I20260810 21:02:06.104583 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786395574307397-15136-0/minicluster-data/master-2/wal/instance
I20260810 21:02:06.104764 15136 external_mini_cluster.cc:1428] Setting key 0d3482d0ad1c6206ae0f9f559905443e
I20260810 21:02:06.105924 29939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395726, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:02:06.115758 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.106655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.14.200.62:42749)
0810 21:02:06.106723 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.106725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.106891 (+ 166us) server_negotiation.cc:407] Connection header received
0810 21:02:06.106958 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.106960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.106972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.107038 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.107483 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.108173 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.109333 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.109612 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.109673 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.111833 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.111835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.111835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.111843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.114441 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.114791 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.114791 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.114792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.114844 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.115126 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.115127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.115127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.115241 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.115317 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.115590 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.115669 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:02:06.116102 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.106600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:42749 (local address 127.0.0.1:34202)
0810 21:02:06.106783 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.106827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.106890 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.107063 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.107065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.107297 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.107393 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.107397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.108257 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.108257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.109263 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.109266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.109697 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.109698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.109772 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.110192 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.110202 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.111747 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.114530 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.114537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.114538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.114726 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.114926 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.114927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.114927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.115045 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.115310 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.115317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.115455 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.115564 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.115712 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260810 21:02:06.116110 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.105309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.14.200.60:37391)
0810 21:02:06.105483 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.105491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.105517 (+ 26us) server_negotiation.cc:407] Connection header received
0810 21:02:06.105904 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.105909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.105950 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.106067 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.106855 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.108078 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.109107 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.109511 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.109619 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.112454 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.112463 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.112463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.112470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.114437 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.115055 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.115058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.115059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.115118 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.115332 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.115333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.115333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.115482 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.115624 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.115712 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.115788 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260810 21:02:06.116688 29939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395726, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:06.123955 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.116984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:38703 (local address 127.0.0.1:48228)
0810 21:02:06.117026 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.117035 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.117069 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.117245 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.117246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.117404 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.117431 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.117432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.118169 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.118170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.118696 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.118698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.119128 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.119129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.119183 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.119721 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.119725 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.121196 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.122916 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.122917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.122918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.123142 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.123389 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.123390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.123390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.123451 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.123739 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.123739 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.123774 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.123821 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.123890 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 21:02:06.124073 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.117068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.14.200.61:38703)
0810 21:02:06.117112 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.117114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.117122 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:02:06.117162 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.117164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.117174 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.117223 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.117506 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.118098 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.118773 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.119060 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.119116 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.121301 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.121302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.121302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.121309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.122822 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.123245 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.123247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.123247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.123315 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.123528 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.123528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.123529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.123665 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.123735 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.123873 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.123953 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:02:06.124428 29939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:02:06.125427 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.117208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.14.200.62:42749)
0810 21:02:06.117243 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.117244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.117251 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:06.117268 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.117269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.117278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.117329 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.117838 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.118313 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.119011 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.119247 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.119337 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.122350 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.122351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.122351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.122358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.123931 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.124617 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.124618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.124618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.124679 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.124934 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.124935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.124936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.125059 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.125228 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.125266 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.125341 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395726, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:02:06.130860 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.124671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54230)
0810 21:02:06.124713 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.124722 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.124755 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.124963 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.124964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.125071 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.125115 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.125117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.125906 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.125907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.126404 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.126406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.126730 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.126731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.126778 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.127183 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.127186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.128290 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.129840 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.129841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.129841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.130051 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.130227 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.130228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.130228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.130295 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.130530 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.130531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.130574 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.130649 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.130734 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 21:02:06.131148 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.124732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.14.200.60:37391)
0810 21:02:06.124764 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.124766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.124803 (+ 37us) server_negotiation.cc:407] Connection header received
0810 21:02:06.124846 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.124847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.124855 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.124945 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.125196 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.125852 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.126478 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.126655 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.126731 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.128384 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.128385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.128385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.128391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.129750 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.130113 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.130114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.130114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.130160 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.130365 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.130366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.130366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.130458 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.130507 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.130973 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.131044 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:02:06.132589 29939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: Bootstrap starting.
I20260810 21:02:06.133517 29939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:06.133643 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.125992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.14.200.61:38703)
0810 21:02:06.126031 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.126033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.126545 (+ 512us) server_negotiation.cc:407] Connection header received
0810 21:02:06.126612 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.126613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.126622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.126674 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.127218 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.127679 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.128411 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.128665 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.128724 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.131116 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.131118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.131118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.131126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.132475 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.132855 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.132856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.132856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.132909 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.133118 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.133119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.133119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.133235 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.133430 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.133466 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.133549 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:02:06.133888 29939 log.cc:824] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:06.134749 29939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: No bootstrap required, opened a new log
I20260810 21:02:06.135278 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:02:06.137043 29939 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.137208 29939 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:06.137261 29939 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd6eafc634644eb9c822f0754749bee, State: Initialized, Role: FOLLOWER
I20260810 21:02:06.137398 29939 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.137681 29952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.137742 29939 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:06.137880 29952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: This master's current role is: FOLLOWER
W20260810 21:02:06.139837 29964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:02:06.139901 29964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:02:06.139940 29864 master_runner.cc:186] Error getting master registration for 127.14.200.62:42749: OK, instance_id { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" instance_seqno: 1786395725802122 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:02:06.142930 29864 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:02:06.145200 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.136925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.14.200.60:37391)
0810 21:02:06.137004 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.137005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.137138 (+ 133us) server_negotiation.cc:407] Connection header received
0810 21:02:06.137197 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.137198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.137208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.137261 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.138725 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.139527 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.140372 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.140653 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.140726 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.142306 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.142307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.142308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.142315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.143917 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.144311 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.144313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.144313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.144367 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.144579 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.144581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.144581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.144740 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.144887 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.145013 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.145087 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 21:02:06.145875 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.136889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.14.200.62:42749)
0810 21:02:06.136962 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.136964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.136978 (+ 14us) server_negotiation.cc:407] Connection header received
0810 21:02:06.137292 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.137293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.137353 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.137388 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.138088 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.138808 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.140256 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.140499 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.140531 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.142813 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.142815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.142815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.142822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.144586 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.145052 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.145053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.145054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.145114 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.145331 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.145332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.145332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.145455 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.145529 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.145731 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.145801 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 21:02:06.146521 29777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } attempt: 2
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:02:06.152392 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.137055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.14.200.61:38703)
0810 21:02:06.137099 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.137101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.137349 (+ 248us) server_negotiation.cc:407] Connection header received
0810 21:02:06.137442 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.137443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.137453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.137506 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.138230 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.138951 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.141312 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.141674 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.141713 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.149812 (+ 8099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.149813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.149813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.149821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.151077 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.151405 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.151406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.151407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.151479 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.151784 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.151785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.151786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.151921 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.152056 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.152209 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.152295 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:02:06.154091 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.146809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.14.200.60:37391)
0810 21:02:06.146848 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.146851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.146870 (+ 19us) server_negotiation.cc:407] Connection header received
0810 21:02:06.146887 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.146888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.146896 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.146941 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.147183 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.147614 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.148232 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.148426 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.148507 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.151449 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.151450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.151450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.151457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.153090 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.153443 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.153444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.153444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.153491 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.153677 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.153678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.153678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.153767 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.153833 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.153984 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.154038 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:02:06.154170 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.146737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54252)
0810 21:02:06.146783 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.146793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.146829 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.146946 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.146946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.147081 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.147111 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.147113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.147690 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.147691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.148154 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.148155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.148509 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.148510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.148563 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.149025 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.149029 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.151070 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.153161 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.153162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.153162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.153370 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.153555 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.153556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.153556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.153621 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.153847 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.153848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.153893 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.153971 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.154065 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 21:02:06.155901 29777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: Bootstrap starting.
I20260810 21:02:06.156687 29777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:06.157043 29777 log.cc:824] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:06.157931 29777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: No bootstrap required, opened a new log
I20260810 21:02:06.159359 29777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.159502 29777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:06.159550 29777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82afff2b34c34efa99d31d9e5737be75, State: Initialized, Role: FOLLOWER
I20260810 21:02:06.159672 29777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.159951 29967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.160005 29777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:06.160051 29967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 21:02:06.161846 29980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:02:06.161900 29980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:02:06.162650 29705 master_runner.cc:186] Error getting master registration for 127.14.200.61:38703: OK, instance_id { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" instance_seqno: 1786395725956897 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:02:06.165511 29705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:02:06.190688 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } attempt: 3
Aug 10 21:02:06 dist-test-slave-1x4g krb5kdc[29692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395725, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:02:06.199016 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.191016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37391 (local address 127.0.0.1:54258)
0810 21:02:06.191067 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:06.191079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:06.191117 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:06.191304 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:06.191305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:06.191525 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:06.191585 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.191587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.192544 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.192546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.193369 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:06.193371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.193887 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.193888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:06.193955 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.194468 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:06.194474 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:06.195961 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:06.197851 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.197852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.197852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.198071 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.198314 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:06.198314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:06.198315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:06.198382 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:06.198670 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:06.198671 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:06.198717 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 21:02:06.198798 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 21:02:06.198889 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 21:02:06.199028 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:06.191076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.14.200.60:37391)
0810 21:02:06.191178 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:06.191180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:06.191188 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:02:06.191212 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:06.191213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:06.191222 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:06.191286 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:06.191663 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.192454 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.193455 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:06.193802 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:06.193874 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:06.196050 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:06.196052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:06.196053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:06.196060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:06.197766 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.198161 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.198162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.198163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.198240 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:06.198453 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:06.198454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:06.198454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:06.198586 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:06.198654 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:06.198834 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 21:02:06.198909 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 21:02:06.201022 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: Bootstrap starting.
I20260810 21:02:06.201719 29857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:06.201985 29857 log.cc:824] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:06.202863 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: No bootstrap required, opened a new log
I20260810 21:02:06.204967 29857 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.205119 29857 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:06.205149 29857 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35ed759a15d42c4b72a163125c64be8, State: Initialized, Role: FOLLOWER
I20260810 21:02:06.205271 29857 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.205605 29857 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 21:02:06.208341 29995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:02:06.208446 29995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:02:06.205562 29983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.209746 29983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:02:06.210665 29784 master_runner.cc:263] No leader master found from master c35ed759a15d42c4b72a163125c64be8
I20260810 21:02:06.210722 29784 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 21:02:06.299882 29727 tablet.cc:2369] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:06.349200 29885 tablet.cc:2369] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:06.454838 29807 tablet.cc:2369] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:06.492527 29952 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:06.492677 29952 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.492998 29952 leader_election.cc:288] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82afff2b34c34efa99d31d9e5737be75 (127.14.200.62:42749), c35ed759a15d42c4b72a163125c64be8 (127.14.200.61:38703)
I20260810 21:02:06.493287 29749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd6eafc634644eb9c822f0754749bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82afff2b34c34efa99d31d9e5737be75" is_pre_election: true
I20260810 21:02:06.493310 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd6eafc634644eb9c822f0754749bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c35ed759a15d42c4b72a163125c64be8" is_pre_election: true
I20260810 21:02:06.493445 29749 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdd6eafc634644eb9c822f0754749bee in term 0.
I20260810 21:02:06.493445 29830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdd6eafc634644eb9c822f0754749bee in term 0.
I20260810 21:02:06.493631 29883 leader_election.cc:302] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c35ed759a15d42c4b72a163125c64be8, cdd6eafc634644eb9c822f0754749bee; no voters:
I20260810 21:02:06.493831 29952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:06.493911 29952 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:06.493949 29952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:06.494695 29952 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.494837 29952 leader_election.cc:288] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [CANDIDATE]: Term 1 election: Requested vote from peers 82afff2b34c34efa99d31d9e5737be75 (127.14.200.62:42749), c35ed759a15d42c4b72a163125c64be8 (127.14.200.61:38703)
I20260810 21:02:06.495033 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd6eafc634644eb9c822f0754749bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c35ed759a15d42c4b72a163125c64be8"
I20260810 21:02:06.495100 29749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdd6eafc634644eb9c822f0754749bee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82afff2b34c34efa99d31d9e5737be75"
I20260810 21:02:06.495126 29830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:06.495182 29749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:06.495975 29830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdd6eafc634644eb9c822f0754749bee in term 1.
I20260810 21:02:06.495975 29749 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdd6eafc634644eb9c822f0754749bee in term 1.
I20260810 21:02:06.496124 29883 leader_election.cc:302] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c35ed759a15d42c4b72a163125c64be8, cdd6eafc634644eb9c822f0754749bee; no voters:
I20260810 21:02:06.496215 29952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:06.496318 29952 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [term 1 LEADER]: Becoming Leader. State: Replica: cdd6eafc634644eb9c822f0754749bee, State: Running, Role: LEADER
I20260810 21:02:06.496451 29952 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } }
I20260810 21:02:06.496937 29997 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd6eafc634644eb9c822f0754749bee. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.497045 29997 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:06.500901 30000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:06.501120 30000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:06.502555 29830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [term 1 FOLLOWER]: Refusing update from remote peer cdd6eafc634644eb9c822f0754749bee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:06.502583 29749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [term 1 FOLLOWER]: Refusing update from remote peer cdd6eafc634644eb9c822f0754749bee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:06.502861 29997 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [LEADER]: Connected to new peer: Peer: permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:06.502949 29952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [LEADER]: Connected to new peer: Peer: permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:06.504040 29983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd6eafc634644eb9c822f0754749bee. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.504141 29983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:02:06.504225 29967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd6eafc634644eb9c822f0754749bee. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.504325 29967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:02:06.505077 29952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.505168 29952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:06.505151 29997 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.505208 29997 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:06.505407 30000 catalog_manager.cc:1347] Generated new cluster ID: 8472fb86fc3e4813bf6e85a33a35773a
I20260810 21:02:06.505398 30002 mvcc.cc:205] Tried to move back new op lower bound from 7317076895752380416 to 7317076895730036736. Current Snapshot: MvccSnapshot[applied={T|T < 7317076895752380416}]
I20260810 21:02:06.505501 30000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:02:06.505492 29983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.505569 29983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:02:06.505681 29967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdd6eafc634644eb9c822f0754749bee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82afff2b34c34efa99d31d9e5737be75" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 42749 } } peers { permanent_uuid: "c35ed759a15d42c4b72a163125c64be8" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 38703 } } peers { permanent_uuid: "cdd6eafc634644eb9c822f0754749bee" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37391 } } }
I20260810 21:02:06.505760 29967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:02:06.505960 30003 mvcc.cc:205] Tried to move back new op lower bound from 7317076895752380416 to 7317076895730036736. Current Snapshot: MvccSnapshot[applied={T|T < 7317076895752380416 or (T in {7317076895752380416})}]
I20260810 21:02:06.531785 30000 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:02:06.532434 30000 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:06.540606 30000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdd6eafc634644eb9c822f0754749bee: Generated new TSK 0
I20260810 21:02:06.540799 30000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:02:07.162336 29980 catalog_manager.cc:1259] Loaded cluster ID: 8472fb86fc3e4813bf6e85a33a35773a
I20260810 21:02:07.162431 29980 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: loading cluster ID for follower catalog manager: success
I20260810 21:02:07.163281 29980 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: acquiring CA information for follower catalog manager: success
I20260810 21:02:07.163601 29980 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82afff2b34c34efa99d31d9e5737be75: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:02:07.208976 29995 catalog_manager.cc:1259] Loaded cluster ID: 8472fb86fc3e4813bf6e85a33a35773a
I20260810 21:02:07.209084 29995 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: loading cluster ID for follower catalog manager: success
I20260810 21:02:07.209919 29995 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: acquiring CA information for follower catalog manager: success
I20260810 21:02:07.210259 29995 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c35ed759a15d42c4b72a163125c64be8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:02:07.607713 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29705
I20260810 21:02:07.615401 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29784
I20260810 21:02:07.621165 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 29864
2026-08-10T21:02:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3142 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:02:07 dist-test-slave-1x4g krb5kdc[30018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:02:07 dist-test-slave-1x4g krb5kdc[30018](info): set up 2 sockets
Aug 10 21:02:07 dist-test-slave-1x4g krb5kdc[30018](info): commencing operation
krb5kdc: starting...
W20260810 21:02:08.659979 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:02:08 dist-test-slave-1x4g krb5kdc[30018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:02:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:02:08.777653 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33793
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:33793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:08.861974 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:08.862202 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:08.862223 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:08.863276 30031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:08.863308 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:08.863322 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:08.863333 30031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:08.863344 30031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:08.864702 30031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:33793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:33793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:08.864954 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:08.865136 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:08.869911 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:08.869940 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:08.869951 30031 server_base.cc:1034] running on GCE node
W20260810 21:02:08.869920 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:08.870426 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:08.870755 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:08.871912 30031 hybrid_clock.cc:648] HybridClock initialized: now 1786395728871898 us; error 26 us; skew 500 ppm
Aug 10 21:02:08 dist-test-slave-1x4g krb5kdc[30018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395728, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:08.874330 30031 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:08.874786 30031 webserver.cc:460] Webserver started at http://127.14.200.62:33729/ using document root <none> and password file <none>
I20260810 21:02:08.875010 30031 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:08.875077 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:08.875227 30031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:08.876238 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "5797d7e566584795ab3d160a9754d686"
format_stamp: "Formatted at 2026-08-10 21:02:08 on dist-test-slave-1x4g"
server_key: "c5f38c4472d064e5e72c41b6ea20e8d7"
server_key_iv: "0ce722e8a71441b4e02706c3b5db2e1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:08.876608 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "5797d7e566584795ab3d160a9754d686"
format_stamp: "Formatted at 2026-08-10 21:02:08 on dist-test-slave-1x4g"
server_key: "c5f38c4472d064e5e72c41b6ea20e8d7"
server_key_iv: "0ce722e8a71441b4e02706c3b5db2e1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:08.877993 30031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:08.878695 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:08.878859 30031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:08.878962 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "5797d7e566584795ab3d160a9754d686"
format_stamp: "Formatted at 2026-08-10 21:02:08 on dist-test-slave-1x4g"
server_key: "c5f38c4472d064e5e72c41b6ea20e8d7"
server_key_iv: "0ce722e8a71441b4e02706c3b5db2e1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:08.879041 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:08.906324 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:08.908464 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:08.908632 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:08.912708 30031 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:33793
I20260810 21:02:08.912755 30101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:33793 every 8 connection(s)
I20260810 21:02:08.913151 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:08.913954 30102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:08.916252 30102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686: Bootstrap starting.
I20260810 21:02:08.916891 30102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:08.917200 30102 log.cc:824] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:08.917896 30102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686: No bootstrap required, opened a new log
I20260810 21:02:08.919457 30102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } }
I20260810 21:02:08.919590 30102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:08.919638 30102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5797d7e566584795ab3d160a9754d686, State: Initialized, Role: FOLLOWER
I20260810 21:02:08.919780 30102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } }
I20260810 21:02:08.919842 30102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:08.919862 30102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:08.919929 30102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:08.920446 30102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } }
I20260810 21:02:08.920567 30102 leader_election.cc:302] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5797d7e566584795ab3d160a9754d686; no voters:
I20260810 21:02:08.920753 30102 leader_election.cc:288] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:02:08.920828 30107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:08.921015 30107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [term 1 LEADER]: Becoming Leader. State: Replica: 5797d7e566584795ab3d160a9754d686, State: Running, Role: LEADER
I20260810 21:02:08.921066 30102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:08.921176 30107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } }
I20260810 21:02:08.921545 30109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5797d7e566584795ab3d160a9754d686. Latest consensus state: current_term: 1 leader_uuid: "5797d7e566584795ab3d160a9754d686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } } }
I20260810 21:02:08.921628 30109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:08.921670 30108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5797d7e566584795ab3d160a9754d686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5797d7e566584795ab3d160a9754d686" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 33793 } } }
I20260810 21:02:08.921729 30108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:08.921931 30116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:08.922465 30116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:08.923172 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30031
I20260810 21:02:08.923282 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:02:08.923491 15136 external_mini_cluster.cc:1428] Setting key efd9a66e58fa4ecfcd066b9cc00ac2fd
I20260810 21:02:08.924039 30116 catalog_manager.cc:1347] Generated new cluster ID: 5f6f5b8cf44d4f4a96ea8afbfc2af0c9
I20260810 21:02:08.924093 30116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:02:08 dist-test-slave-1x4g krb5kdc[30018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:08.933683 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:08.923959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.14.200.62:33793)
0810 21:02:08.925136 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:08.925143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:08.925166 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:02:08.925272 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:08.925277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:08.925324 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:08.925451 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:08.926126 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:08.927044 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:08.927788 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:08.928172 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:08.928282 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:08.930435 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:08.930439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:08.930439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:08.930457 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:08.932091 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:08.932479 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:08.932484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:08.932485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:08.932576 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:08.932803 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:08.932804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:08.932804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:08.933042 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:08.933211 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:08.933318 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 21:02:08.933462 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1139,"thread_start_us":103,"threads_started":1}
I20260810 21:02:08.934639 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:02:08.943959 30116 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:02:08.944659 30116 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:08.951180 30116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5797d7e566584795ab3d160a9754d686: Generated new TSK 0
I20260810 21:02:08.951344 30116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:02:08.988539 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30031
2026-08-10T21:02:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1359 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:02:09 dist-test-slave-1x4g krb5kdc[30133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:02:09 dist-test-slave-1x4g krb5kdc[30133](info): set up 2 sockets
Aug 10 21:02:09 dist-test-slave-1x4g krb5kdc[30133](info): commencing operation
krb5kdc: starting...
W20260810 21:02:10.051087 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:02:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:02:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:02:10.168138 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:10.247720 30146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:10.247942 30146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:10.247973 30146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:10.249199 30146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:10.249266 30146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:10.249282 30146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:10.249341 30146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:10.249361 30146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:10.250833 30146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:10.251055 30146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:10.251269 30146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:10.256150 30155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.256197 30152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.256150 30153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:10.256297 30146 server_base.cc:1034] running on GCE node
I20260810 21:02:10.256608 30146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:10.256989 30146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:10.258157 30146 hybrid_clock.cc:648] HybridClock initialized: now 1786395730258146 us; error 29 us; skew 500 ppm
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:10.260634 30146 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:10.261130 30146 webserver.cc:460] Webserver started at http://127.14.200.62:35271/ using document root <none> and password file <none>
I20260810 21:02:10.261368 30146 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:10.261435 30146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:10.261569 30146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:10.262549 30146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "f3ecbede11bf411fa04c0442336904a2"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "7cf45569307490baa1015af83eea1ccb"
server_key_iv: "cdae850a001378cd85484eee6ee7938d"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.262894 30146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "f3ecbede11bf411fa04c0442336904a2"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "7cf45569307490baa1015af83eea1ccb"
server_key_iv: "cdae850a001378cd85484eee6ee7938d"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.264278 30146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:02:10.265059 30162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.265228 30146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:10.265345 30146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f3ecbede11bf411fa04c0442336904a2"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "7cf45569307490baa1015af83eea1ccb"
server_key_iv: "cdae850a001378cd85484eee6ee7938d"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.265415 30146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:10.283898 30146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:10.285738 30146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:10.285900 30146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:10.290082 30146 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36959
I20260810 21:02:10.290112 30219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36959 every 8 connection(s)
I20260810 21:02:10.290433 30146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:10.291375 30220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:10.292982 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30146
I20260810 21:02:10.293100 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:02:10.293311 15136 external_mini_cluster.cc:1428] Setting key 56de7f431a5eba908b2b70d214c036e1
I20260810 21:02:10.293893 30220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap starting.
I20260810 21:02:10.294512 30220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:10.294816 30220 log.cc:824] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:10.295485 30220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: No bootstrap required, opened a new log
I20260810 21:02:10.297242 30220 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:10.297392 30220 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:10.297441 30220 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Initialized, Role: FOLLOWER
I20260810 21:02:10.297571 30220 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:10.297673 30220 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:10.297732 30220 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:10.297783 30220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:10.298485 30220 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:10.298607 30220 leader_election.cc:302] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ecbede11bf411fa04c0442336904a2; no voters:
I20260810 21:02:10.298784 30220 leader_election.cc:288] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:02:10.298822 30225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:10.299005 30225 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 LEADER]: Becoming Leader. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Running, Role: LEADER
I20260810 21:02:10.299057 30220 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:10.299154 30225 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:10.299575 30226 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:10.299674 30226 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:10.299583 30227 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ecbede11bf411fa04c0442336904a2. Latest consensus state: current_term: 1 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:10.299777 30227 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:10.299948 30234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:10.300453 30234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:10.301945 30234 catalog_manager.cc:1347] Generated new cluster ID: ce15eb6d15f447e3aafa1b1dcefdcf0f
I20260810 21:02:10.301996 30234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:10.307521 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.293851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.14.200.62:36959)
0810 21:02:10.294026 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.294033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.294058 (+ 25us) server_negotiation.cc:407] Connection header received
0810 21:02:10.294498 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.294505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.294571 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.294728 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:10.295405 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.296300 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.297137 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.297622 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.297744 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.304402 (+ 6658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:10.304406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:10.304406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:10.304418 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:10.306043 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.306417 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.306420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.306421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.306512 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.306725 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.306726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.306726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.306943 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:10.307112 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.307207 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.307343 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260810 21:02:10.343855 30234 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:02:10.344631 30234 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:10.357537 30234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Generated new TSK 0
I20260810 21:02:10.357729 30234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:02:10.394995 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:10.477794 30247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:10.478078 30247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:10.478106 30247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:10.479781 30247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:10.479857 30247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:10.479897 30247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:10.481616 30247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:10.481868 30247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:10.482182 30247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:10.488191 30252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:10.488230 30247 server_base.cc:1034] running on GCE node
W20260810 21:02:10.488327 30255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.488603 30253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:10.488866 30247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:10.489151 30247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:10.490329 30247 hybrid_clock.cc:648] HybridClock initialized: now 1786395730490314 us; error 27 us; skew 500 ppm
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:10.493494 30247 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:10.493927 30247 webserver.cc:460] Webserver started at http://127.14.200.1:40263/ using document root <none> and password file <none>
I20260810 21:02:10.494184 30247 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:10.494253 30247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:10.494405 30247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:10.495585 30247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "c931513400d34c2ea21dd0b799184d12"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "d482f3035d5ab0529d7fb5e63f9ca75a"
server_key_iv: "bc5c605db790e3fc13c03af4bc707a1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.495919 30247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "c931513400d34c2ea21dd0b799184d12"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "d482f3035d5ab0529d7fb5e63f9ca75a"
server_key_iv: "bc5c605db790e3fc13c03af4bc707a1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.497675 30247 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 21:02:10.498571 30262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.498765 30247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:10.498927 30247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "c931513400d34c2ea21dd0b799184d12"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "d482f3035d5ab0529d7fb5e63f9ca75a"
server_key_iv: "bc5c605db790e3fc13c03af4bc707a1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.499008 30247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:10.509176 30247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:10.511531 30247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:10.511694 30247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:10.511955 30247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:10.512308 30247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:10.512359 30247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.512404 30247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:10.512437 30247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.520757 30247 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:33663
I20260810 21:02:10.520833 30382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:33663 every 8 connection(s)
I20260810 21:02:10.521189 30247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:10.529440 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.521936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:41035 (local address 127.14.200.62:36959)
0810 21:02:10.522000 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.522001 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.522186 (+ 185us) server_negotiation.cc:407] Connection header received
0810 21:02:10.522627 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.522628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.522637 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.522690 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:10.523467 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.523900 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.524878 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.525136 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.525195 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.527117 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:10.527118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:10.527118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:10.527125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:10.528376 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.528685 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.528686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.528686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.528734 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.528972 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.528972 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.528972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.529067 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:10.529116 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.529321 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.529381 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:02:10.529572 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.521912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:41035)
0810 21:02:10.522072 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.522110 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.522552 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.522740 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.522743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.523240 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:10.523401 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.523405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.523976 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.523977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.524744 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.524745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.525215 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.525216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.525330 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.525710 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:10.525717 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:10.527043 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:10.528439 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.528444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.528446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.528629 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.528828 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.528829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.528829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.528919 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.529121 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:10.529126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:10.529223 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.529298 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.529402 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260810 21:02:10.529876 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30247
I20260810 21:02:10.529971 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:02:10.530056 30383 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:10.530145 30383 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:10.530208 15136 external_mini_cluster.cc:1428] Setting key fea8d92977709a78b7559fcc15b68d70
I20260810 21:02:10.530319 30383 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:10.530824 30180 ts_manager.cc:194] Registered new tserver with Master: c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663)
I20260810 21:02:10.531658 30180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:41035
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:02:10.555109 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:10.635001 30389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:10.635190 30389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:10.635213 30389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:10.636180 30389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:10.636211 30389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:10.636234 30389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:10.637669 30389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:10.637840 30389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:10.638007 30389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:10.642714 30399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.642747 30396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.642730 30397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:10.642874 30389 server_base.cc:1034] running on GCE node
I20260810 21:02:10.643141 30389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:10.643353 30389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:10.644484 30389 hybrid_clock.cc:648] HybridClock initialized: now 1786395730644463 us; error 37 us; skew 500 ppm
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:10.647166 30389 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:10.647667 30389 webserver.cc:460] Webserver started at http://127.14.200.2:44869/ using document root <none> and password file <none>
I20260810 21:02:10.647892 30389 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:10.647967 30389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:10.648092 30389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:10.649371 30389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "912595a9ec3c16af8ef584c9a8eb7074"
server_key_iv: "b214257dc9fd77b59b1c8b7ad5b2e0a3"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.649709 30389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "912595a9ec3c16af8ef584c9a8eb7074"
server_key_iv: "b214257dc9fd77b59b1c8b7ad5b2e0a3"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.650822 30389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:10.651441 30406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.651608 30389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.651702 30389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "912595a9ec3c16af8ef584c9a8eb7074"
server_key_iv: "b214257dc9fd77b59b1c8b7ad5b2e0a3"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.651783 30389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:10.674957 30389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:10.676717 30389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:10.676913 30389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:10.677145 30389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:10.677456 30389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:10.677511 30389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.677558 30389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:10.677587 30389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.683358 30389 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46475
I20260810 21:02:10.683409 30526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46475 every 8 connection(s)
I20260810 21:02:10.683705 30389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:10.690177 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30389
I20260810 21:02:10.690308 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:02:10.690558 15136 external_mini_cluster.cc:1428] Setting key bb0fbf83c6163c85a4dfaee382c15a5e
I20260810 21:02:10.692745 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.684418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:33375 (local address 127.14.200.62:36959)
0810 21:02:10.684475 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.684477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.684707 (+ 230us) server_negotiation.cc:407] Connection header received
0810 21:02:10.685317 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.685318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.685330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.685392 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:10.686302 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.686773 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.687677 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.687875 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.687945 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.689818 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:10.689820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:10.689820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:10.689828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:10.691334 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.691805 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.691806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.691806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.691871 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.692111 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.692112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.692112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.692231 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:10.692300 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.692571 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.692653 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 21:02:10.692931 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.684413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:33375)
0810 21:02:10.684559 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.684613 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.685191 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.685465 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.685469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.686002 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:10.686198 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.686202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.686865 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.686867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.687588 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.687590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.687954 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.687955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.688074 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.688441 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:10.688448 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:10.689721 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:10.691452 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.691460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.691462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.691714 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.691935 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.691936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.691936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.692040 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.692303 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:10.692310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:10.692428 (+ 118us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.692537 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.692694 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260810 21:02:10.693466 30527 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:10.693571 30527 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:10.693739 30527 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:10.694150 30180 ts_manager.cc:194] Registered new tserver with Master: 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475)
I20260810 21:02:10.694507 30180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:33375
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:02:10.715915 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:10.795943 30532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:10.796118 30532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:10.796139 30532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:10.797667 30532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:10.797739 30532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:10.797785 30532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:10.799254 30532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:10.799540 30532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:10.799822 30532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:10.804823 30537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.804809 30540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:10.804806 30538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:10.805047 30532 server_base.cc:1034] running on GCE node
I20260810 21:02:10.805251 30532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:10.805548 30532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:10.806704 30532 hybrid_clock.cc:648] HybridClock initialized: now 1786395730806687 us; error 34 us; skew 500 ppm
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:10.814033 30532 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:10.814428 30532 webserver.cc:460] Webserver started at http://127.14.200.3:42325/ using document root <none> and password file <none>
I20260810 21:02:10.814671 30532 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:10.814718 30532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:10.814883 30532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:10.815883 30532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "2c66541e546ca2571751a10eab015354"
server_key_iv: "ecbbde680d86541c1085db4148fee3ca"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.816212 30532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "2c66541e546ca2571751a10eab015354"
server_key_iv: "ecbbde680d86541c1085db4148fee3ca"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.817559 30532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:02:10.818240 30549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.818439 30532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:10.818535 30532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "2c66541e546ca2571751a10eab015354"
server_key_iv: "ecbbde680d86541c1085db4148fee3ca"
server_key_version: "encryptionkey@0"
I20260810 21:02:10.818605 30532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:10.849275 30532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:10.851130 30532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:10.851290 30532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:10.851524 30532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:10.851824 30532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:10.851856 30532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.851897 30532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:10.851922 30532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:10.858472 30532 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:45601
I20260810 21:02:10.858530 30672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:45601 every 8 connection(s)
I20260810 21:02:10.858832 30532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:10.860884 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30532
I20260810 21:02:10.860957 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:02:10.861124 15136 external_mini_cluster.cc:1428] Setting key 064c7e347e46887d3d7b8b24812b797e
Aug 10 21:02:10 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395730, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:10.867488 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.859489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:43231 (local address 127.14.200.62:36959)
0810 21:02:10.859558 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.859561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.859739 (+ 178us) server_negotiation.cc:407] Connection header received
0810 21:02:10.860213 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.860214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.860223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.860270 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:10.861245 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.861856 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.862811 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.863026 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.863109 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.865133 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:10.865134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:10.865135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:10.865142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:10.866218 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.866594 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.866595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.866596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.866656 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.866897 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.866898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.866898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.867007 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:10.867082 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.867330 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.867406 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:02:10.867669 30675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.859469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:43231)
0810 21:02:10.859626 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.859660 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.860120 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.860320 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.860324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.860917 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:10.861162 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.861169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.861947 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.861949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.862720 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.862723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.863123 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.863124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.863259 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.863695 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:10.863703 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:10.865042 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:10.866306 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.866313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.866314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.866523 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.866725 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:10.866726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:10.866726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:10.866834 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:10.867083 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:10.867091 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:10.867214 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.867309 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.867446 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":99,"threads_started":1}
I20260810 21:02:10.868196 30673 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:10.868268 30673 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:10.868436 30673 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:10.868795 30180 ts_manager.cc:194] Registered new tserver with Master: ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
I20260810 21:02:10.869232 30180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:43231
I20260810 21:02:10.872912 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:10.879348 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.874179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.14.200.62:36959)
0810 21:02:10.874250 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.874251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.874346 (+ 95us) server_negotiation.cc:407] Connection header received
0810 21:02:10.874384 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.874384 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.874392 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.874447 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:10.874965 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.875411 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.876067 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.876222 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.876270 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.877338 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:10.877339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:10.877339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:10.877345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:10.878469 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.878757 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.878757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.878758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.878804 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:10.878966 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:10.878966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:10.878967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:10.879086 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:10.879153 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.879240 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.879293 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 21:02:10.880733 30180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:02:10.882359 30180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:02:10.888839 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.885135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.14.200.3:45601)
0810 21:02:10.885349 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.885353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.885362 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:02:10.885396 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.885398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.885413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.885488 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:10.885678 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886676 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.887728 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.888427 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.888505 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.888567 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.888708 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.888745 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260810 21:02:10.889045 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.884854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.14.200.2:46475)
0810 21:02:10.885369 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.885373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.885385 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:02:10.885423 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.885425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.885440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.885515 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:10.885740 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886407 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.887811 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.888500 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.888577 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.888627 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.888915 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.888976 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":85,"threads_started":1}
I20260810 21:02:10.888871 30684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.884706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46475 (local address 127.0.0.1:51168)
0810 21:02:10.885003 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.885012 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.885056 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.885600 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.885600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.885602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:10.885656 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.885658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886513 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.886514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.887733 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.887736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.888582 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.888583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.888631 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.888636 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.888683 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.888760 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":6,"thread_start_us":111,"threads_started":1}
I20260810 21:02:10.888873 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.884924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:45601 (local address 127.0.0.1:44796)
0810 21:02:10.885148 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.885157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.885210 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.885559 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.885560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.885562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:10.885605 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.885606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886763 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.886764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.887648 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.887651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.888509 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.888510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.888593 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.888608 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.888683 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.888760 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":34,"thread_start_us":93,"threads_started":1}
I20260810 21:02:10.889863 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.884670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.0.0.1:38236)
0810 21:02:10.884866 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:10.884905 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:10.884972 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:10.885283 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:10.885286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:10.885288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:10.885422 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.885424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886723 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.886724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.887748 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:10.887750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.889616 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.889617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:10.889687 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.889691 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:10.889747 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 21:02:10.889793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":41,"threads_started":1}
I20260810 21:02:10.889889 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:10.884813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.14.200.1:33663)
0810 21:02:10.885081 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:10.885086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:10.885097 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:02:10.885142 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:10.885144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:10.885163 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:10.885248 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:10.885528 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.886615 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.888689 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:10.889441 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:10.889512 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:10.889578 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:10.889776 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 21:02:10.889818 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":93,"threads_started":1}
I20260810 21:02:10.890054 30459 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 (DEFAULT_TABLE table=test-table [id=ebcb6db44144416d9c9b1061081c8f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:10.890426 30459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1a8db78fcb4ce78a57c80445665064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:10.890612 30316 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 (DEFAULT_TABLE table=test-table [id=ebcb6db44144416d9c9b1061081c8f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:10.890913 30316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1a8db78fcb4ce78a57c80445665064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:10.893157 30601 tablet_service.cc:1467] Processing CreateTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 (DEFAULT_TABLE table=test-table [id=ebcb6db44144416d9c9b1061081c8f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:10.893381 30601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c1a8db78fcb4ce78a57c80445665064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:10.894035 30693 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Bootstrap starting.
I20260810 21:02:10.894248 30694 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Bootstrap starting.
I20260810 21:02:10.894747 30693 tablet_bootstrap.cc:654] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:10.894788 30694 tablet_bootstrap.cc:654] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:10.895203 30694 log.cc:824] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:10.895358 30693 log.cc:824] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:10.895989 30694 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: No bootstrap required, opened a new log
I20260810 21:02:10.896060 30694 ts_tablet_manager.cc:1397] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:02:10.896106 30693 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: No bootstrap required, opened a new log
I20260810 21:02:10.896163 30693 ts_tablet_manager.cc:1397] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:02:10.896720 30698 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Bootstrap starting.
I20260810 21:02:10.897383 30698 tablet_bootstrap.cc:654] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:10.897943 30698 log.cc:824] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:10.897890 30694 raft_consensus.cc:348] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.898037 30694 raft_consensus.cc:374] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:10.898082 30694 raft_consensus.cc:729] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931513400d34c2ea21dd0b799184d12, State: Initialized, Role: FOLLOWER
I20260810 21:02:10.898200 30694 consensus_queue.cc:260] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.898391 30693 raft_consensus.cc:348] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.898496 30694 ts_tablet_manager.cc:1428] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:02:10.898546 30693 raft_consensus.cc:374] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:10.898589 30693 raft_consensus.cc:729] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cdc7cbcb434cdaa6abdc1567c67ba8, State: Initialized, Role: FOLLOWER
I20260810 21:02:10.898681 30698 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: No bootstrap required, opened a new log
I20260810 21:02:10.898680 30383 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
I20260810 21:02:10.898746 30698 ts_tablet_manager.cc:1397] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:10.898700 30693 consensus_queue.cc:260] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.898976 30693 ts_tablet_manager.cc:1428] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:02:10.899065 30527 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
I20260810 21:02:10.900534 30698 raft_consensus.cc:348] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.900687 30698 raft_consensus.cc:374] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:10.900708 30698 raft_consensus.cc:729] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab92e9344ab64eb88e6ee05c66c2c9b5, State: Initialized, Role: FOLLOWER
I20260810 21:02:10.900817 30698 consensus_queue.cc:260] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:10.901031 30698 ts_tablet_manager.cc:1428] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 21:02:10.901086 30673 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
W20260810 21:02:10.934234 30528 tablet.cc:2369] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:11.021972 30384 tablet.cc:2369] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:11.071475 30703 raft_consensus.cc:482] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:11.071669 30703 raft_consensus.cc:504] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:11.072117 30703 leader_election.cc:288] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
I20260810 21:02:11.075547 30705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.072332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:45601 (local address 127.14.200.2:51265)
0810 21:02:11.072490 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.072499 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.072545 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.072712 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.072713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.072715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.072819 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.072821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.073791 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.073792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.074671 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.074674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.075250 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.075251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.075345 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.075348 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.075420 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.075457 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260810 21:02:11.075533 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.072447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:51265 (local address 127.14.200.3:45601)
0810 21:02:11.072496 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.072498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.072541 (+ 43us) server_negotiation.cc:407] Connection header received
0810 21:02:11.072614 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.072615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.072627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.072688 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.072913 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.073699 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.074765 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.075164 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.075253 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.075263 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.075428 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.075470 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 21:02:11.075867 30622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1a8db78fcb4ce78a57c80445665064" candidate_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" is_pre_election: true
I20260810 21:02:11.076097 30622 raft_consensus.cc:2454] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69cdc7cbcb434cdaa6abdc1567c67ba8 in term 0.
I20260810 21:02:11.076303 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.072294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.14.200.2:55739)
0810 21:02:11.072415 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.072426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.072485 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.073749 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.073750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.073752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.073827 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.073829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.074623 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.074624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.075311 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.075313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.075993 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.075994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.076117 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.076120 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.076189 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.076236 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":21}
I20260810 21:02:11.076313 30687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.073560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:55739 (local address 127.14.200.1:33663)
0810 21:02:11.073628 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.073630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.073638 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:02:11.073662 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.073662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.073672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.073731 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.073895 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.074544 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.075413 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.075906 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.075985 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.075993 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.076195 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.076243 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 21:02:11.076333 30407 leader_election.cc:302] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5; no voters:
I20260810 21:02:11.076458 30703 raft_consensus.cc:2781] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:11.076502 30703 raft_consensus.cc:482] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:11.076537 30703 raft_consensus.cc:3037] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.076663 30337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1a8db78fcb4ce78a57c80445665064" candidate_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931513400d34c2ea21dd0b799184d12" is_pre_election: true
I20260810 21:02:11.076815 30337 raft_consensus.cc:2454] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69cdc7cbcb434cdaa6abdc1567c67ba8 in term 0.
I20260810 21:02:11.077437 30703 raft_consensus.cc:504] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:11.077554 30703 leader_election.cc:288] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [CANDIDATE]: Term 1 election: Requested vote from peers c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
I20260810 21:02:11.077755 30337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1a8db78fcb4ce78a57c80445665064" candidate_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931513400d34c2ea21dd0b799184d12"
I20260810 21:02:11.077850 30337 raft_consensus.cc:3037] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.078505 30337 raft_consensus.cc:2454] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cdc7cbcb434cdaa6abdc1567c67ba8 in term 1.
I20260810 21:02:11.078624 30622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c1a8db78fcb4ce78a57c80445665064" candidate_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
I20260810 21:02:11.078692 30410 leader_election.cc:302] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, c931513400d34c2ea21dd0b799184d12; no voters:
I20260810 21:02:11.078709 30622 raft_consensus.cc:3037] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.078807 30703 raft_consensus.cc:2781] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:11.078981 30703 raft_consensus.cc:686] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 69cdc7cbcb434cdaa6abdc1567c67ba8, State: Running, Role: LEADER
I20260810 21:02:11.079079 30703 consensus_queue.cc:237] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:11.079690 30622 raft_consensus.cc:2454] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cdc7cbcb434cdaa6abdc1567c67ba8 in term 1.
I20260810 21:02:11.080235 30180 catalog_manager.cc:5495] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } health_report { overall_health: UNKNOWN } } }
W20260810 21:02:11.109514 30674 tablet.cc:2369] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:11.113721 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.111640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.14.200.2:46475)
0810 21:02:11.111707 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.111709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.111716 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:11.111750 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.111751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.111766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.111824 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:11.111947 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.112367 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.113076 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.113257 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.113308 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.113440 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:11.113537 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:11.113595 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.113633 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.113651 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:02:11.115527 30337 raft_consensus.cc:1261] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Refusing update from remote peer 69cdc7cbcb434cdaa6abdc1567c67ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:11.115531 30622 raft_consensus.cc:1261] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Refusing update from remote peer 69cdc7cbcb434cdaa6abdc1567c67ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:11.115863 30703 consensus_queue.cc:1035] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:11.116030 30706 consensus_queue.cc:1035] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:11.119243 30712 mvcc.cc:205] Tried to move back new op lower bound from 7317076914646478848 to 7317076914500517888. Current Snapshot: MvccSnapshot[applied={T|T < 7317076914646478848}]
I20260810 21:02:11.121165 30180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:11.121263 30180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:11.122071 30180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Sending DeleteTablet for 3 replicas of tablet 1c1a8db78fcb4ce78a57c80445665064
I20260810 21:02:11.122434 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30247
I20260810 21:02:11.122479 30316 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38236
I20260810 21:02:11.122956 30459 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51168
I20260810 21:02:11.123588 30721 tablet_replica.cc:330] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: stopping tablet replica
I20260810 21:02:11.123718 30721 raft_consensus.cc:2229] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:02:11.123853 30721 raft_consensus.cc:2258] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:11.124622 30601 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:44796
I20260810 21:02:11.124890 30721 ts_tablet_manager.cc:1905] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:11.124910 30722 tablet_replica.cc:330] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: stopping tablet replica
I20260810 21:02:11.125000 30722 raft_consensus.cc:2229] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:11.125106 30722 raft_consensus.cc:2258] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:11.125553 30722 ts_tablet_manager.cc:1905] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:11.126906 30721 ts_tablet_manager.cc:1918] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:11.127032 30721 log.cc:1196] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/1c1a8db78fcb4ce78a57c80445665064
I20260810 21:02:11.127398 30722 ts_tablet_manager.cc:1918] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:11.127494 30722 log.cc:1196] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/1c1a8db78fcb4ce78a57c80445665064
I20260810 21:02:11.127630 30721 ts_tablet_manager.cc:1939] T 1c1a8db78fcb4ce78a57c80445665064 P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting consensus metadata
I20260810 21:02:11.127813 30722 ts_tablet_manager.cc:1939] T 1c1a8db78fcb4ce78a57c80445665064 P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting consensus metadata
I20260810 21:02:11.128144 30163 catalog_manager.cc:4841] TS 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): tablet 1c1a8db78fcb4ce78a57c80445665064 (table test-table [id=ebcb6db44144416d9c9b1061081c8f35]) successfully deleted
I20260810 21:02:11.128237 30163 catalog_manager.cc:4841] TS ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): tablet 1c1a8db78fcb4ce78a57c80445665064 (table test-table [id=ebcb6db44144416d9c9b1061081c8f35]) successfully deleted
W20260810 21:02:11.131790 30167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:33663 (error 108)
I20260810 21:02:11.132054 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30389
I20260810 21:02:11.132831 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.132148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.0.0.1:38252)
0810 21:02:11.132721 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.132761 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:33663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550}
W20260810 21:02:11.133002 30167 catalog_manager.cc:4584] TS c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c1a8db78fcb4ce78a57c80445665064: Network error: Client connection negotiation failed: client connection to 127.14.200.1:33663: connect: Connection refused (error 111)
I20260810 21:02:11.138312 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30532
I20260810 21:02:11.143927 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30146
I20260810 21:02:11.150417 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--webserver_interface=127.14.200.62
--webserver_port=35271
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:11.233121 30724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:11.233357 30724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:11.233397 30724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:11.234540 30724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:11.234611 30724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:11.234638 30724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:11.234663 30724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:11.234699 30724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:11.236207 30724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=35271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:11.236505 30724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:11.236783 30724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:11.247957 30730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.248088 30731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.247957 30734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:11.248255 30724 server_base.cc:1034] running on GCE node
I20260810 21:02:11.248560 30724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:11.248962 30724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:11.250128 30724 hybrid_clock.cc:648] HybridClock initialized: now 1786395731250117 us; error 38 us; skew 500 ppm
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:11.252533 30724 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:11.253033 30724 webserver.cc:460] Webserver started at http://127.14.200.62:35271/ using document root <none> and password file <none>
I20260810 21:02:11.253224 30724 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:11.253278 30724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:11.254657 30724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:11.255327 30741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.255519 30724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:11.255599 30724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f3ecbede11bf411fa04c0442336904a2"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "7cf45569307490baa1015af83eea1ccb"
server_key_iv: "cdae850a001378cd85484eee6ee7938d"
server_key_version: "encryptionkey@0"
I20260810 21:02:11.255945 30724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:11.265215 30724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:11.266973 30724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:11.267141 30724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:11.270967 30724 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36959
I20260810 21:02:11.271016 30797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36959 every 8 connection(s)
I20260810 21:02:11.271389 30724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:11.273106 30798 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:02:11.273765 30798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap starting.
I20260810 21:02:11.275658 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30724
I20260810 21:02:11.275995 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:33663
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=40263
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:11.283298 30798 log.cc:824] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:11.285243 30798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:11.285614 30798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap complete.
I20260810 21:02:11.287907 30798 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.288123 30798 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Initialized, Role: FOLLOWER
I20260810 21:02:11.288300 30798 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.288359 30798 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:11.288411 30798 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:11.288460 30798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:02:11.289503 30798 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.289628 30798 leader_election.cc:302] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ecbede11bf411fa04c0442336904a2; no voters:
I20260810 21:02:11.289832 30798 leader_election.cc:288] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 21:02:11.289867 30802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 21:02:11.289970 30802 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 LEADER]: Becoming Leader. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Running, Role: LEADER
I20260810 21:02:11.290119 30798 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:11.290096 30802 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.290459 30804 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:11.290469 30805 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ecbede11bf411fa04c0442336904a2. Latest consensus state: current_term: 2 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:11.290616 30804 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:11.290632 30805 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:11.291115 30810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:11.291932 30810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:11.292050 30810 catalog_manager.cc:1259] Loaded cluster ID: ce15eb6d15f447e3aafa1b1dcefdcf0f
I20260810 21:02:11.292088 30810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:02:11.292315 30810 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 21:02:11.296986 30810 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 21:02:11.299324 30810 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:11.299567 30810 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Loaded TSK: 0
I20260810 21:02:11.299793 30810 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:02:11.359984 30800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:11.360194 30800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:11.360231 30800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:11.361445 30800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:11.361506 30800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:11.361560 30800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:11.362926 30800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=40263
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:11.363204 30800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:11.363431 30800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:11.369078 30830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.369074 30828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.369108 30827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:11.369246 30800 server_base.cc:1034] running on GCE node
I20260810 21:02:11.369477 30800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:11.369750 30800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:11.370898 30800 hybrid_clock.cc:648] HybridClock initialized: now 1786395731370881 us; error 34 us; skew 500 ppm
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:11.373451 30800 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:11.374039 30800 webserver.cc:460] Webserver started at http://127.14.200.1:40263/ using document root <none> and password file <none>
I20260810 21:02:11.374333 30800 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:11.374392 30800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:11.375733 30800 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:11.376461 30837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.376621 30800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:11.376730 30800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "c931513400d34c2ea21dd0b799184d12"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "d482f3035d5ab0529d7fb5e63f9ca75a"
server_key_iv: "bc5c605db790e3fc13c03af4bc707a1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:11.377106 30800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:11.386559 30800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:11.388623 30800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:11.388760 30800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:11.389042 30800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:11.389539 30844 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 21:02:11.391199 30800 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 21:02:11.391248 30800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 21:02:11.391281 30800 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 21:02:11.392004 30800 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 21:02:11.392050 30800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:11.392091 30844 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Bootstrap starting.
I20260810 21:02:11.399923 30800 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:33663
I20260810 21:02:11.400416 30800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:11.400965 30961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:33663 every 8 connection(s)
I20260810 21:02:11.401726 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30800
I20260810 21:02:11.402050 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:46475
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=44869
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:11.403916 30844 log.cc:824] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:11.407672 30844 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:11.407951 30844 tablet_bootstrap.cc:492] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Bootstrap complete.
I20260810 21:02:11.409034 30844 ts_tablet_manager.cc:1397] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260810 21:02:11.410261 30844 raft_consensus.cc:348] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:11.410586 30844 raft_consensus.cc:729] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931513400d34c2ea21dd0b799184d12, State: Initialized, Role: FOLLOWER
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:11.416661 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.401160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:53929)
0810 21:02:11.401533 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.403997 (+ 2464us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.404625 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.405310 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.405332 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.405715 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:11.405962 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.405968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.407614 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.407615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.408860 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.408865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.409662 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.409664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.409810 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.410424 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:11.410434 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:11.412210 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:11.414733 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.414741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.414743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.415080 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.415403 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.415403 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.415404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.415532 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.416026 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:11.416032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:11.416173 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.416281 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.416436 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":84,"threads_started":1}
I20260810 21:02:11.416843 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.401526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53929 (local address 127.14.200.62:36959)
0810 21:02:11.404927 (+ 3401us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.404940 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.404970 (+ 30us) server_negotiation.cc:407] Connection header received
0810 21:02:11.405074 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.405078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.405126 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.405280 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:11.406100 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.407477 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.408965 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.409548 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.409657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.412328 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:11.412332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:11.412332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:11.412345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:11.414632 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.415177 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.415180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.415181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.415322 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.415614 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.415615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.415615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.415934 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:11.416027 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.416369 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.416537 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3354,"thread_start_us":100,"threads_started":1}
I20260810 21:02:11.416879 30844 consensus_queue.cc:260] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } } peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } }
I20260810 21:02:11.417619 30844 ts_tablet_manager.cc:1428] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260810 21:02:11.417889 30962 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:11.417992 30962 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:11.418162 30962 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:11.418933 30758 ts_manager.cc:194] Registered new tserver with Master: c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663)
I20260810 21:02:11.420189 30758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:53929
I20260810 21:02:11.421202 30962 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
I20260810 21:02:11.427320 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.419576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.0.0.1:38258)
0810 21:02:11.424182 (+ 4606us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.424227 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.424283 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.424537 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.424540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.424542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.424629 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.424631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.425527 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.425529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.426334 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.426336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.426992 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.426993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.427101 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.427120 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.427200 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.427255 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4584,"thread_start_us":87,"threads_started":1}
I20260810 21:02:11.427316 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.419806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.14.200.1:33663)
0810 21:02:11.424380 (+ 4574us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.424383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.424390 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:11.424427 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.424428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.424438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.424526 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.424720 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.425430 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.426437 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.426897 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.426989 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.427062 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.427217 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.427255 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4549,"thread_start_us":65,"threads_started":1}
I20260810 21:02:11.427681 30869 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c1a8db78fcb4ce78a57c80445665064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:38258
I20260810 21:02:11.428141 30972 tablet_replica.cc:330] stopping tablet replica
I20260810 21:02:11.428243 30972 raft_consensus.cc:2229] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:11.428308 30972 raft_consensus.cc:2258] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:11.428606 30972 ts_tablet_manager.cc:1905] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:11.430841 30972 ts_tablet_manager.cc:1918] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:11.430902 30972 log.cc:1196] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/1c1a8db78fcb4ce78a57c80445665064
I20260810 21:02:11.431128 30972 ts_tablet_manager.cc:1939] T 1c1a8db78fcb4ce78a57c80445665064 P c931513400d34c2ea21dd0b799184d12: Deleting consensus metadata
I20260810 21:02:11.431578 30745 catalog_manager.cc:4841] TS c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663): tablet 1c1a8db78fcb4ce78a57c80445665064 (table test-table [id=ebcb6db44144416d9c9b1061081c8f35]) successfully deleted
W20260810 21:02:11.492082 30966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:11.492295 30966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:11.492331 30966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:11.493476 30966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:11.493532 30966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:11.493572 30966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:11.495020 30966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:46475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=44869
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:11.495281 30966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:11.495533 30966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:11.500371 30980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.500403 30977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.500417 30978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:11.500627 30966 server_base.cc:1034] running on GCE node
I20260810 21:02:11.500855 30966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:11.501098 30966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:11.502251 30966 hybrid_clock.cc:648] HybridClock initialized: now 1786395731502237 us; error 31 us; skew 500 ppm
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:11.504741 30966 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:11.505206 30966 webserver.cc:460] Webserver started at http://127.14.200.2:44869/ using document root <none> and password file <none>
I20260810 21:02:11.505434 30966 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:11.505513 30966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:11.506796 30966 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:11.507339 30988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.507501 30966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:11.507606 30966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "912595a9ec3c16af8ef584c9a8eb7074"
server_key_iv: "b214257dc9fd77b59b1c8b7ad5b2e0a3"
server_key_version: "encryptionkey@0"
I20260810 21:02:11.507928 30966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:11.523592 30966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:11.525789 30966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:11.525964 30966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:11.526219 30966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:11.526528 30966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:11.526582 30966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.526621 30966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:11.526667 30966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.532369 30966 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46475
I20260810 21:02:11.532420 31115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46475 every 8 connection(s)
I20260810 21:02:11.532729 30966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:02:11.534154 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 30966
I20260810 21:02:11.534487 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:45601
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=42325
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:11.544497 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.533505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:48571)
0810 21:02:11.533658 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.533699 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.534381 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.534697 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.534702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.535565 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:11.535847 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.535855 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.536602 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.536603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.537957 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.537961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.538505 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.538506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.538607 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.539193 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:11.539201 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:11.540990 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:11.542960 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.542967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.542968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.543182 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.543443 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.543445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.543445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.543574 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.543956 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:11.543963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:11.544078 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.544178 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.544325 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260810 21:02:11.544500 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.533528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:48571 (local address 127.14.200.62:36959)
0810 21:02:11.533593 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.533595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.533788 (+ 193us) server_negotiation.cc:407] Connection header received
0810 21:02:11.534513 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.534515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.534529 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.534618 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:11.535969 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.536467 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.538064 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.538411 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.538500 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.541099 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:11.541101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:11.541101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:11.541111 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:11.542873 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.543261 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.543263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.543263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.543336 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.543659 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.543660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.543661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.543825 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:11.543927 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.544250 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.544360 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 21:02:11.545225 31116 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:11.545344 31116 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:11.545568 31116 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:11.546015 30758 ts_manager.cc:194] Registered new tserver with Master: 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475)
I20260810 21:02:11.546377 30758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:48571
W20260810 21:02:11.621311 31119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:11.621587 31119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:11.621629 31119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:11.622756 31119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:11.622826 31119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:11.622875 31119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:11.624396 31119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:45601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=42325
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:11.624683 31119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:11.625023 31119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:11.630156 31127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.630266 31125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.630339 31124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:11.630288 31119 server_base.cc:1034] running on GCE node
I20260810 21:02:11.630615 31119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:11.630854 31119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:11.632014 31119 hybrid_clock.cc:648] HybridClock initialized: now 1786395731631996 us; error 30 us; skew 500 ppm
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:11.634497 31119 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:11.634943 31119 webserver.cc:460] Webserver started at http://127.14.200.3:42325/ using document root <none> and password file <none>
I20260810 21:02:11.635144 31119 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:11.635196 31119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:11.636418 31119 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:11.637053 31134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.637226 31119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:11.637312 31119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "2c66541e546ca2571751a10eab015354"
server_key_iv: "ecbbde680d86541c1085db4148fee3ca"
server_key_version: "encryptionkey@0"
I20260810 21:02:11.637605 31119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:11.647851 31119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:11.649658 31119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:11.649799 31119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:11.650012 31119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:11.650352 31119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:11.650413 31119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.650444 31119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:11.650470 31119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.656483 31119 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:45601
I20260810 21:02:11.656579 31262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:45601 every 8 connection(s)
I20260810 21:02:11.656952 31119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:11.661203 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31119
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:11.666440 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.657656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:60509 (local address 127.14.200.62:36959)
0810 21:02:11.657754 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.657756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.657938 (+ 182us) server_negotiation.cc:407] Connection header received
0810 21:02:11.658477 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.658478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.658491 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.658543 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:11.659421 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.659884 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.660887 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.661180 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.661256 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.663240 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:11.663242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:11.663242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:11.663249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:11.664871 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.665242 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.665243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.665243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.665306 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.665811 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.665811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.665812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.665915 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:11.665994 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.666286 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.666357 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 21:02:11.666627 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.657631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:60509)
0810 21:02:11.657836 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.657867 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.658381 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.658602 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.658606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.659146 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:11.659336 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.659342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.659975 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.659976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.660752 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.660754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.661277 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.661279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.661461 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.661910 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:11.661921 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:11.663160 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:11.664949 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.664956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.664957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.665162 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.665391 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:11.665392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:11.665392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:11.665565 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:11.665993 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:11.666001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:11.666150 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.666261 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.666400 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260810 21:02:11.667083 31263 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36959
I20260810 21:02:11.667182 31263 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:11.667347 31263 heartbeater.cc:511] Master 127.14.200.62:36959 requested a full tablet report, sending...
I20260810 21:02:11.667747 30758 ts_manager.cc:194] Registered new tserver with Master: ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
I20260810 21:02:11.668087 30758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:60509
I20260810 21:02:11.668231 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.661585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.14.200.62:36959)
0810 21:02:11.661870 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.661872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.661878 (+ 6us) server_negotiation.cc:407] Connection header received
0810 21:02:11.661898 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.661899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.661910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.661978 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:11.662473 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.662930 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.663788 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.664013 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.664078 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.665316 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:11.665317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:11.665318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:11.665323 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:11.667130 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.667449 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.667450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.667450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.667505 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:11.667716 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:11.667718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:11.667718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:11.667836 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:11.668000 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.668044 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.668120 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":82,"threads_started":1}
I20260810 21:02:11.669014 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:11.671121 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.669318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.14.200.62:36959)
0810 21:02:11.669348 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.669349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.669438 (+ 89us) server_negotiation.cc:407] Connection header received
0810 21:02:11.669500 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.669501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.669525 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.669569 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:11.669632 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.670009 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.670601 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.670761 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.670805 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.670914 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:11.670991 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:11.671038 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.671070 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.671085 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 21:02:11.671347 30758 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:02:11.672880 30758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:02:11.674993 30869 tablet_service.cc:1467] Processing CreateTablet for tablet 44b018aa2e144a36bdcff69ccd5d493b (DEFAULT_TABLE table=test-table [id=810e73a16f03483bac82b5281969a26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:11.675191 30869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b018aa2e144a36bdcff69ccd5d493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:11.676853 30844 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Bootstrap starting.
I20260810 21:02:11.677170 30844 tablet_bootstrap.cc:654] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:11.677765 30844 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: No bootstrap required, opened a new log
I20260810 21:02:11.677819 30844 ts_tablet_manager.cc:1397] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:11.677922 30844 raft_consensus.cc:348] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.677995 30844 raft_consensus.cc:374] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:11.678043 30844 raft_consensus.cc:729] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931513400d34c2ea21dd0b799184d12, State: Initialized, Role: FOLLOWER
I20260810 21:02:11.678094 30844 consensus_queue.cc:260] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.678187 30844 ts_tablet_manager.cc:1428] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:11.679049 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.675118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.14.200.3:45601)
0810 21:02:11.675261 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.675265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.675276 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:02:11.675314 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.675316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.675332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.675459 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.675641 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.676654 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.678036 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.678706 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.678800 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.678853 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.678944 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.678980 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260810 21:02:11.679064 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.674756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46475 (local address 127.0.0.1:51174)
0810 21:02:11.674791 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.674801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.674834 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.675498 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.675499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.675500 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.675536 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.675537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.676508 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.676509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.677629 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.677632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.678795 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.678796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.678857 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.678861 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.678925 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.678971 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 21:02:11.679090 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.674940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.14.200.2:46475)
0810 21:02:11.675300 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.675304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.675314 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:11.675350 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.675353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.675369 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.675652 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.675799 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.676413 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.677705 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.678721 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.678793 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.678847 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.678966 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.679006 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":82,"threads_started":1}
I20260810 21:02:11.679068 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.674852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:45601 (local address 127.0.0.1:44800)
0810 21:02:11.675027 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.675038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.675178 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.675461 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.675462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.675464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.675560 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.675562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.676776 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.676777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.677961 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.677963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.678805 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.678806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.678870 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.678873 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.678930 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.678980 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260810 21:02:11.679715 31041 tablet_service.cc:1467] Processing CreateTablet for tablet 44b018aa2e144a36bdcff69ccd5d493b (DEFAULT_TABLE table=test-table [id=810e73a16f03483bac82b5281969a26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:11.679781 31191 tablet_service.cc:1467] Processing CreateTablet for tablet 44b018aa2e144a36bdcff69ccd5d493b (DEFAULT_TABLE table=test-table [id=810e73a16f03483bac82b5281969a26d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:11.680099 31041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b018aa2e144a36bdcff69ccd5d493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:11.680114 31191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b018aa2e144a36bdcff69ccd5d493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:11.682763 31273 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Bootstrap starting.
I20260810 21:02:11.682930 31274 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Bootstrap starting.
I20260810 21:02:11.683377 31273 tablet_bootstrap.cc:654] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:11.683426 31274 tablet_bootstrap.cc:654] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:11.683971 31274 log.cc:824] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:11.684098 31273 log.cc:824] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:11.684834 31274 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: No bootstrap required, opened a new log
I20260810 21:02:11.684897 31273 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: No bootstrap required, opened a new log
I20260810 21:02:11.684912 31274 ts_tablet_manager.cc:1397] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:02:11.684983 31273 ts_tablet_manager.cc:1397] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:02:11.686743 31274 raft_consensus.cc:348] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.686743 31273 raft_consensus.cc:348] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.686918 31274 raft_consensus.cc:374] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:11.686918 31273 raft_consensus.cc:374] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:11.686954 31274 raft_consensus.cc:729] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cdc7cbcb434cdaa6abdc1567c67ba8, State: Initialized, Role: FOLLOWER
I20260810 21:02:11.686965 31273 raft_consensus.cc:729] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab92e9344ab64eb88e6ee05c66c2c9b5, State: Initialized, Role: FOLLOWER
I20260810 21:02:11.687071 31273 consensus_queue.cc:260] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.687088 31274 consensus_queue.cc:260] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.687351 31274 ts_tablet_manager.cc:1428] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:11.687364 31263 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
I20260810 21:02:11.687371 31116 heartbeater.cc:503] Master 127.14.200.62:36959 was elected leader, sending a full tablet report...
I20260810 21:02:11.687351 31273 ts_tablet_manager.cc:1428] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:11.715341 31279 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:11.715423 31279 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.715720 31279 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663)
I20260810 21:02:11.718541 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.715861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46475 (local address 127.14.200.3:47401)
0810 21:02:11.715916 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.715929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.715977 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.716189 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.716190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.716191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.716272 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.716274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.716974 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.716975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.717718 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.717720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.718287 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.718288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.718371 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.718375 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.718428 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.718459 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 21:02:11.718561 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.715933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:47401 (local address 127.14.200.2:46475)
0810 21:02:11.715974 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.715976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.715984 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:02:11.716084 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.716085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.716096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.716153 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.716352 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.716886 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.717819 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.718212 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.718264 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.718271 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.718453 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.718498 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:02:11.718647 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.715992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:42745 (local address 127.14.200.1:33663)
0810 21:02:11.716051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.716053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.716171 (+ 118us) server_negotiation.cc:407] Connection header received
0810 21:02:11.716199 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.716200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.716209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.716267 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:11.716452 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.716970 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.717912 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.718286 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.718355 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.718363 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.718527 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.718573 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:02:11.718851 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.715897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.14.200.3:42745)
0810 21:02:11.716076 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.716086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:11.716146 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:11.716278 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:11.716279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:11.716281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:11.716365 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.716366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.717068 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.717069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.717832 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:11.717834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.718373 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.718374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:11.718448 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.718452 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:11.718507 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 21:02:11.718551 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":90,"threads_started":1}
I20260810 21:02:11.718864 31062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b018aa2e144a36bdcff69ccd5d493b" candidate_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" is_pre_election: true
I20260810 21:02:11.718984 31062 raft_consensus.cc:2454] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab92e9344ab64eb88e6ee05c66c2c9b5 in term 0.
I20260810 21:02:11.719239 31135 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5; no voters:
I20260810 21:02:11.719381 31279 raft_consensus.cc:2781] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:11.719372 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b018aa2e144a36bdcff69ccd5d493b" candidate_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931513400d34c2ea21dd0b799184d12" is_pre_election: true
I20260810 21:02:11.719435 31279 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:11.719491 31279 raft_consensus.cc:3037] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.719527 30909 raft_consensus.cc:2454] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab92e9344ab64eb88e6ee05c66c2c9b5 in term 0.
I20260810 21:02:11.720176 31279 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.720315 31279 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [CANDIDATE]: Term 1 election: Requested vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663)
I20260810 21:02:11.720460 31062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b018aa2e144a36bdcff69ccd5d493b" candidate_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
I20260810 21:02:11.720498 30909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b018aa2e144a36bdcff69ccd5d493b" candidate_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c931513400d34c2ea21dd0b799184d12"
I20260810 21:02:11.720538 31062 raft_consensus.cc:3037] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.720584 30909 raft_consensus.cc:3037] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:11.721369 31062 raft_consensus.cc:2454] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab92e9344ab64eb88e6ee05c66c2c9b5 in term 1.
I20260810 21:02:11.721369 30909 raft_consensus.cc:2454] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab92e9344ab64eb88e6ee05c66c2c9b5 in term 1.
I20260810 21:02:11.721553 31138 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab92e9344ab64eb88e6ee05c66c2c9b5, c931513400d34c2ea21dd0b799184d12; no voters:
I20260810 21:02:11.721698 31279 raft_consensus.cc:2781] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:11.721848 31279 raft_consensus.cc:686] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 LEADER]: Becoming Leader. State: Replica: ab92e9344ab64eb88e6ee05c66c2c9b5, State: Running, Role: LEADER
I20260810 21:02:11.721943 31279 consensus_queue.cc:237] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:11.722733 30757 catalog_manager.cc:5495] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } health_report { overall_health: UNKNOWN } } }
I20260810 21:02:11.737615 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.735260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.14.200.3:45601)
0810 21:02:11.735310 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:11.735312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:11.735322 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:11.735410 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:11.735411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:11.735427 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:11.735509 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:11.735633 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.736081 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.736841 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:11.737030 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:11.737105 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:11.737258 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:11.737351 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:11.737434 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:11.737493 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 21:02:11.737513 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 21:02:11.739264 31062 raft_consensus.cc:1261] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Refusing update from remote peer ab92e9344ab64eb88e6ee05c66c2c9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:11.739295 30909 raft_consensus.cc:1261] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Refusing update from remote peer ab92e9344ab64eb88e6ee05c66c2c9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:11.739598 31281 consensus_queue.cc:1035] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:11.739727 31279 consensus_queue.cc:1035] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:11.743882 30756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:11.743971 30756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:11.744715 30756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Sending DeleteTablet for 3 replicas of tablet 44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:11.745108 31191 tablet_service.cc:1514] Processing DeleteTablet for tablet 44b018aa2e144a36bdcff69ccd5d493b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:44800
I20260810 21:02:11.745121 31041 tablet_service.cc:1514] Processing DeleteTablet for tablet 44b018aa2e144a36bdcff69ccd5d493b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:11 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51174
I20260810 21:02:11.745321 31295 tablet_replica.cc:330] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: stopping tablet replica
I20260810 21:02:11.745340 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30800
I20260810 21:02:11.745421 31295 raft_consensus.cc:2229] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:02:11.745510 31295 raft_consensus.cc:2258] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:11.745967 31295 ts_tablet_manager.cc:1905] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:11.748176 31295 ts_tablet_manager.cc:1918] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:11.748271 31295 log.cc:1196] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:11.748597 31295 ts_tablet_manager.cc:1939] T 44b018aa2e144a36bdcff69ccd5d493b P ab92e9344ab64eb88e6ee05c66c2c9b5: Deleting consensus metadata
I20260810 21:02:11.749030 30742 catalog_manager.cc:4841] TS ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): tablet 44b018aa2e144a36bdcff69ccd5d493b (table test-table [id=810e73a16f03483bac82b5281969a26d]) successfully deleted
I20260810 21:02:11.752890 31296 tablet_replica.cc:330] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: stopping tablet replica
I20260810 21:02:11.753014 31296 raft_consensus.cc:2229] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:11.753121 31296 raft_consensus.cc:2258] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:11.753530 31296 ts_tablet_manager.cc:1905] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:11.755160 31296 ts_tablet_manager.cc:1918] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:11.755239 31296 log.cc:1196] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/44b018aa2e144a36bdcff69ccd5d493b
W20260810 21:02:11.755491 30745 connection.cc:535] client connection to 127.14.200.1:33663 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:02:11.755550 31296 ts_tablet_manager.cc:1939] T 44b018aa2e144a36bdcff69ccd5d493b P 69cdc7cbcb434cdaa6abdc1567c67ba8: Deleting consensus metadata
W20260810 21:02:11.755589 30745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:02:11.755978 30742 catalog_manager.cc:4841] TS 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): tablet 44b018aa2e144a36bdcff69ccd5d493b (table test-table [id=810e73a16f03483bac82b5281969a26d]) successfully deleted
I20260810 21:02:11.756191 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30966
I20260810 21:02:11.756466 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:11.756332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:33663 (local address 127.0.0.1:38266)
0810 21:02:11.756390 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:11.756430 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:33663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 21:02:11.757670 30745 catalog_manager.cc:4584] TS c931513400d34c2ea21dd0b799184d12 (127.14.200.1:33663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44b018aa2e144a36bdcff69ccd5d493b: Network error: Client connection negotiation failed: client connection to 127.14.200.1:33663: connect: Connection refused (error 111)
I20260810 21:02:11.765041 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 31119
I20260810 21:02:11.772751 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 30724
I20260810 21:02:11.779315 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--webserver_interface=127.14.200.62
--webserver_port=35271
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:11.857928 31298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:11.858134 31298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:11.858167 31298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:11.859236 31298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:11.859301 31298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:11.859335 31298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:11.859354 31298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:11.859380 31298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:11.860754 31298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=35271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:11.861038 31298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:11.861254 31298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:11.865692 31304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.865772 31305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:11.865700 31307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:11.865811 31298 server_base.cc:1034] running on GCE node
I20260810 21:02:11.866279 31298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:11.866497 31298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:11.867642 31298 hybrid_clock.cc:648] HybridClock initialized: now 1786395731867633 us; error 26 us; skew 500 ppm
Aug 10 21:02:11 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395731, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:11.870126 31298 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:11.870534 31298 webserver.cc:460] Webserver started at http://127.14.200.62:35271/ using document root <none> and password file <none>
I20260810 21:02:11.870746 31298 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:11.870790 31298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:11.872066 31298 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:11.872727 31314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:11.872890 31298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:02:11.872995 31298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f3ecbede11bf411fa04c0442336904a2"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "7cf45569307490baa1015af83eea1ccb"
server_key_iv: "cdae850a001378cd85484eee6ee7938d"
server_key_version: "encryptionkey@0"
I20260810 21:02:11.873296 31298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:11.881235 31298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:11.883325 31298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:11.883430 31298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:11.887518 31298 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36959
I20260810 21:02:11.887542 31371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36959 every 8 connection(s)
I20260810 21:02:11.887894 31298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:11.889458 31372 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:02:11.890100 31372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap starting.
I20260810 21:02:11.894091 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31298
I20260810 21:02:11.894400 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:33663
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=40263
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:11.899755 31372 log.cc:824] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:11.901957 31372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:11.902244 31372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2: Bootstrap complete.
I20260810 21:02:11.904455 31372 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.904691 31372 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Initialized, Role: FOLLOWER
I20260810 21:02:11.904915 31372 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.905025 31372 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:11.905076 31372 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:11.905129 31372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 2 FOLLOWER]: Advancing to term 3
I20260810 21:02:11.906494 31372 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.906610 31372 leader_election.cc:302] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ecbede11bf411fa04c0442336904a2; no voters:
I20260810 21:02:11.906806 31372 leader_election.cc:288] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 21:02:11.906838 31378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 21:02:11.906939 31378 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [term 3 LEADER]: Becoming Leader. State: Replica: f3ecbede11bf411fa04c0442336904a2, State: Running, Role: LEADER
I20260810 21:02:11.907039 31378 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } }
I20260810 21:02:11.907069 31372 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:11.907331 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:11.907330 31378 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ecbede11bf411fa04c0442336904a2. Latest consensus state: current_term: 3 leader_uuid: "f3ecbede11bf411fa04c0442336904a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ecbede11bf411fa04c0442336904a2" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36959 } } }
I20260810 21:02:11.907430 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:11.907447 31378 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ecbede11bf411fa04c0442336904a2 [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:11.907622 31383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:11.908422 31383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:11.908541 31383 catalog_manager.cc:1259] Loaded cluster ID: ce15eb6d15f447e3aafa1b1dcefdcf0f
I20260810 21:02:11.908565 31383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 21:02:11.908893 31383 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd0dfbaf9ad google::LogMessage::Fail() at ??:0
@ 0x7fd0dfbb288a google::LogMessage::SendToLog() at ??:0
@ 0x7fd0dfbaf4de google::LogMessage::Flush() at ??:0
@ 0x7fd0dfbb0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd0e7ad00f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd0e7ad98b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd0e1a7d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd0e1a72954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd0e28d66db start_thread at ??:0
@ 0x7fd0de6ad61f clone at ??:0
W20260810 21:02:12.009610 31374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:12.009883 31374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:12.009924 31374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:12.011552 31374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:12.011615 31374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:12.011653 31374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:12.013792 31374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=40263
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:12.014043 31374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:12.014365 31374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:12.019726 31413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:12.019783 31414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:12.019910 31374 server_base.cc:1034] running on GCE node
W20260810 21:02:12.019989 31416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:12.020248 31374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:12.020512 31374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:12.021673 31374 hybrid_clock.cc:648] HybridClock initialized: now 1786395732021660 us; error 31 us; skew 500 ppm
Aug 10 21:02:12 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395732, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:12.024251 31374 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:12.024665 31374 webserver.cc:460] Webserver started at http://127.14.200.1:40263/ using document root <none> and password file <none>
I20260810 21:02:12.024962 31374 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:12.025031 31374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:12.026499 31374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:12.027271 31424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.027457 31374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:12.027565 31374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "c931513400d34c2ea21dd0b799184d12"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "d482f3035d5ab0529d7fb5e63f9ca75a"
server_key_iv: "bc5c605db790e3fc13c03af4bc707a1a"
server_key_version: "encryptionkey@0"
I20260810 21:02:12.027894 31374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:12.049733 31374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:12.052007 31374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:12.052163 31374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:12.052383 31374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:12.052837 31432 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 21:02:12.054847 31374 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 21:02:12.054904 31374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 21:02:12.054945 31374 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 21:02:12.055562 31374 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 21:02:12.055615 31374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:12.055691 31432 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Bootstrap starting.
I20260810 21:02:12.062865 31374 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:33663
I20260810 21:02:12.063020 31548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:33663 every 8 connection(s)
I20260810 21:02:12.063274 31374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:12.067387 31432 log.cc:824] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:12.070448 31432 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:12.070778 31432 tablet_bootstrap.cc:492] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Bootstrap complete.
I20260810 21:02:12.070945 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31374
I20260810 21:02:12.071226 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:46475
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=44869
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:12.072480 31432 ts_tablet_manager.cc:1397] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260810 21:02:12.074100 31432 raft_consensus.cc:348] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:12.074442 31432 raft_consensus.cc:729] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c931513400d34c2ea21dd0b799184d12, State: Initialized, Role: FOLLOWER
I20260810 21:02:12.074615 31432 consensus_queue.cc:260] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:12.074898 31432 ts_tablet_manager.cc:1428] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260810 21:02:12.142617 31551 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:02:12.064026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:58323)
0810 21:02:12.064201 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.064259 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:12.064698 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:12.142447 (+ 77749us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
W20260810 21:02:12.142890 31549 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36959 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36959: Client connection negotiation failed: client connection to 127.14.200.62:36959: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:02:12.143265 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.143149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:53281)
0810 21:02:12.143203 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.143244 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 21:02:12.143613 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.143523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:55701)
0810 21:02:12.143565 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.143589 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 21:02:12.143714 31549 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:02:12.155652 31553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:12.155879 31553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:12.155916 31553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:12.157143 31553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:12.157200 31553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:12.157246 31553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:12.158667 31553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:46475
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=44869
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:12.158928 31553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:12.159155 31553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:12.163880 31560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:12.163885 31562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:12.163887 31559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:12.164101 31553 server_base.cc:1034] running on GCE node
I20260810 21:02:12.164382 31553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:12.164605 31553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:12.165755 31553 hybrid_clock.cc:648] HybridClock initialized: now 1786395732165729 us; error 42 us; skew 500 ppm
Aug 10 21:02:12 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395732, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:12.168004 31553 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:12.168375 31553 webserver.cc:460] Webserver started at http://127.14.200.2:44869/ using document root <none> and password file <none>
I20260810 21:02:12.168596 31553 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:12.168661 31553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:12.169878 31553 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:12.170455 31569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.170624 31553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:12.170718 31553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "912595a9ec3c16af8ef584c9a8eb7074"
server_key_iv: "b214257dc9fd77b59b1c8b7ad5b2e0a3"
server_key_version: "encryptionkey@0"
I20260810 21:02:12.171037 31553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:12.179028 31553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:12.181213 31553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:12.181375 31553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:12.181587 31553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:12.181872 31553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:12.181931 31553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.181969 31553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:12.181998 31553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.187621 31553 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:46475
I20260810 21:02:12.187729 31693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:46475 every 8 connection(s)
I20260810 21:02:12.188032 31553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:02:12.189041 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.188656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:55481)
0810 21:02:12.188840 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.188916 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
W20260810 21:02:12.189431 31694 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36959 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36959: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
I20260810 21:02:12.189694 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.189608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:37933)
0810 21:02:12.189648 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.189675 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 21:02:12.189971 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.189898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:36117)
0810 21:02:12.189935 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.189956 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 21:02:12.190095 31694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 21:02:12.197741 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31553
I20260810 21:02:12.198027 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:45601
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:36959
--webserver_port=42325
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:12.279256 31697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:12.279452 31697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:12.279474 31697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:12.280573 31697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:12.280613 31697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:12.280661 31697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:12.282179 31697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:45601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=42325
--tserver_master_addrs=127.14.200.62:36959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:12.282344 31697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:12.282577 31697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:12.287351 31704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:12.287393 31708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:12.287367 31705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:12.287544 31697 server_base.cc:1034] running on GCE node
I20260810 21:02:12.287755 31697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:12.287976 31697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:12.289131 31697 hybrid_clock.cc:648] HybridClock initialized: now 1786395732289125 us; error 36 us; skew 500 ppm
Aug 10 21:02:12 dist-test-slave-1x4g krb5kdc[30133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395732, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:12.291481 31697 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:12.291914 31697 webserver.cc:460] Webserver started at http://127.14.200.3:42325/ using document root <none> and password file <none>
I20260810 21:02:12.292140 31697 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:12.292207 31697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:12.293452 31697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:12.293977 31715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.294129 31697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:12.294220 31697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5"
format_stamp: "Formatted at 2026-08-10 21:02:10 on dist-test-slave-1x4g"
server_key: "2c66541e546ca2571751a10eab015354"
server_key_iv: "ecbbde680d86541c1085db4148fee3ca"
server_key_version: "encryptionkey@0"
I20260810 21:02:12.294566 31697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:12.308885 31697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:12.310513 31697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:12.310673 31697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:12.310882 31697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:12.311168 31697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:12.311221 31697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.311255 31697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:12.311282 31697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:12.317267 31697 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:45601
I20260810 21:02:12.317317 31841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:45601 every 8 connection(s)
I20260810 21:02:12.317629 31697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:12.318686 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.318342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:35369)
0810 21:02:12.318492 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.318562 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260810 21:02:12.319109 31842 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36959 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36959: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
I20260810 21:02:12.319382 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.319284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:51165)
0810 21:02:12.319314 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.319339 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260810 21:02:12.319669 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:12.319618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:50265)
0810 21:02:12.319638 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:12.319657 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260810 21:02:12.319780 31842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 21:02:12.324156 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31697
I20260810 21:02:13.144538 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.144145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:51407)
0810 21:02:13.144419 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:13.144473 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":81,"threads_started":1}
I20260810 21:02:13.190560 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.190381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:42089)
0810 21:02:13.190487 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:13.190520 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260810 21:02:13.320344 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.320158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:46763)
0810 21:02:13.320269 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:13.320297 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260810 21:02:13.514360 31850 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:13.514506 31850 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:13.514902 31850 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
Aug 10 21:02:13 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395732, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.2@KRBTEST.COM
Aug 10 21:02:13 dist-test-slave-1x4g krb5kdc[30133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395732, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.3@KRBTEST.COM
I20260810 21:02:13.525154 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.515078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:46475 (local address 127.14.200.1:39527)
0810 21:02:13.515154 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:13.515171 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:13.515253 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:13.516174 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:13.516177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:13.516581 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:13.516759 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:13.516765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.517893 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.517894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:13.518683 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:13.518686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.519366 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.519367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:13.519463 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:13.520091 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:13.520095 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:13.521537 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:13.523412 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:13.523419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:13.523421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:13.523632 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:13.523935 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:13.523936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:13.523936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:13.524075 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:13.524548 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:13.524554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:13.524684 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 21:02:13.524812 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 21:02:13.525005 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":74}
I20260810 21:02:13.525167 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.515210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:39527 (local address 127.14.200.2:46475)
0810 21:02:13.515369 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:13.515384 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:13.515422 (+ 38us) server_negotiation.cc:407] Connection header received
0810 21:02:13.515927 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:13.515929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:13.515977 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:13.516091 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:13.516921 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.517777 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.518770 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.519265 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.519384 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:13.521645 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:13.521649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:13.521649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:13.521661 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:13.523315 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:13.523716 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:13.523719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:13.523719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:13.523856 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:13.524145 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:13.524145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:13.524146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:13.524476 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:13.524538 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:13.524890 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 21:02:13.525053 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260810 21:02:13.525245 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.515339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37347 (local address 127.14.200.3:45601)
0810 21:02:13.515508 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:13.515518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:13.515563 (+ 45us) server_negotiation.cc:407] Connection header received
0810 21:02:13.516003 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:13.516005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:13.516071 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:13.516211 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:13.516935 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.517774 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.518786 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.519254 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.519372 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:13.521989 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:13.521993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:13.521993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:13.522006 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:13.523716 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:13.524078 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:13.524081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:13.524081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:13.524190 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:13.524403 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:13.524404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:13.524404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:13.524684 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:13.524792 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:13.524996 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 21:02:13.525122 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260810 21:02:13.525229 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:13.515211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:45601 (local address 127.14.200.1:37347)
0810 21:02:13.515353 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:13.515362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:13.515425 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:13.516224 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:13.516226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:13.516567 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:13.516759 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:13.516765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.517904 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.517905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:13.518656 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:13.518658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:13.519341 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:13.519342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:13.519462 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:13.520029 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:13.520037 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:13.521865 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:13.523787 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:13.523788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:13.523788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:13.524004 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:13.524255 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:13.524256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:13.524256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:13.524319 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:13.524760 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:13.524761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:13.524843 (+ 82us) client_negotiation.cc:769] Sending connection context
0810 21:02:13.524918 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 21:02:13.525085 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":61,"thread_start_us":62,"threads_started":1}
W20260810 21:02:13.526582 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
W20260810 21:02:13.526749 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:13.526791 31425 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c931513400d34c2ea21dd0b799184d12; no voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5
I20260810 21:02:13.526898 31850 raft_consensus.cc:2726] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:02:14.145299 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:14.145059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:59297)
0810 21:02:14.145210 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:14.145251 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260810 21:02:14.191100 31857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:14.190937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:49799)
0810 21:02:14.191040 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:14.191070 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260810 21:02:14.321086 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:14.320765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:42125)
0810 21:02:14.321008 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:14.321052 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260810 21:02:15.146277 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:15.145968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:39601)
0810 21:02:15.146178 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:15.146229 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":65,"threads_started":1}
I20260810 21:02:15.191717 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:15.191528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:39117)
0810 21:02:15.191651 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:15.191681 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260810 21:02:15.321717 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:15.321534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:43113)
0810 21:02:15.321647 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:15.321680 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260810 21:02:15.325954 31863 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:15.326054 31863 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:15.326228 31863 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
W20260810 21:02:15.326634 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
W20260810 21:02:15.326699 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:15.326714 31425 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c931513400d34c2ea21dd0b799184d12; no voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5
I20260810 21:02:15.326776 31863 raft_consensus.cc:2726] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:02:16.147148 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:16.146928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:37871)
0810 21:02:16.147067 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:16.147102 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
I20260810 21:02:16.192313 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:16.192135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:37821)
0810 21:02:16.192250 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:16.192277 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260810 21:02:16.322382 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:16.322183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:59465)
0810 21:02:16.322308 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:16.322344 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260810 21:02:17.148033 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:17.147791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:32789)
0810 21:02:17.147945 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:17.147986 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260810 21:02:17.192983 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:17.192732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:39289)
0810 21:02:17.192914 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:17.192949 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260810 21:02:17.323073 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:17.322889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:38197)
0810 21:02:17.323001 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:17.323035 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260810 21:02:17.844424 31871 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:17.844534 31871 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:17.844728 31871 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
W20260810 21:02:17.845157 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
W20260810 21:02:17.845216 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:17.845232 31425 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c931513400d34c2ea21dd0b799184d12; no voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5
I20260810 21:02:17.845319 31871 raft_consensus.cc:2726] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:02:18.148903 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:18.148638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:40535)
0810 21:02:18.148814 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:18.148853 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260810 21:02:18.193681 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:18.193483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:32819)
0810 21:02:18.193611 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:18.193645 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260810 21:02:18.323827 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:18.323493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:36959)
0810 21:02:18.323750 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:18.323788 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":81,"threads_started":1}
I20260810 21:02:19.149596 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:19.149388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:33445)
0810 21:02:19.149507 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:19.149549 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260810 21:02:19.194242 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:19.194058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:33745)
0810 21:02:19.194174 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:19.194205 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260810 21:02:19.324571 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:19.324272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:33137)
0810 21:02:19.324503 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:19.324536 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":54,"threads_started":1}
I20260810 21:02:20.150300 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:20.150066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:45349)
0810 21:02:20.150219 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:20.150251 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260810 21:02:20.194864 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:20.194634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:50831)
0810 21:02:20.194753 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:20.194800 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260810 21:02:20.325317 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:20.325092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:32999)
0810 21:02:20.325246 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:20.325281 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260810 21:02:21.151162 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:21.150926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:59995)
0810 21:02:21.151075 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:21.151115 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260810 21:02:21.195472 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:21.195284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:35289)
0810 21:02:21.195404 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:21.195437 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260810 21:02:21.325971 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:21.325779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:37983)
0810 21:02:21.325899 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:21.325933 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260810 21:02:22.152037 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:22.151799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:43987)
0810 21:02:22.151942 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:22.151982 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260810 21:02:22.196154 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:22.195951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:47179)
0810 21:02:22.196073 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:22.196115 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260810 21:02:22.326685 31887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:22.326485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:49097)
0810 21:02:22.326614 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:22.326648 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260810 21:02:22.527586 31890 raft_consensus.cc:482] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:22.527707 31890 raft_consensus.cc:504] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cdc7cbcb434cdaa6abdc1567c67ba8" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 46475 } } peers { permanent_uuid: "ab92e9344ab64eb88e6ee05c66c2c9b5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 45601 } } peers { permanent_uuid: "c931513400d34c2ea21dd0b799184d12" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 33663 } }
I20260810 21:02:22.527899 31890 leader_election.cc:288] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475), ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601)
W20260810 21:02:22.528333 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab92e9344ab64eb88e6ee05c66c2c9b5 (127.14.200.3:45601): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
W20260810 21:02:22.528421 31425 leader_election.cc:341] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69cdc7cbcb434cdaa6abdc1567c67ba8 (127.14.200.2:46475): Not found: Tablet not found: 44b018aa2e144a36bdcff69ccd5d493b
I20260810 21:02:22.528458 31425 leader_election.cc:302] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c931513400d34c2ea21dd0b799184d12; no voters: 69cdc7cbcb434cdaa6abdc1567c67ba8, ab92e9344ab64eb88e6ee05c66c2c9b5
I20260810 21:02:22.528545 31890 raft_consensus.cc:2726] T 44b018aa2e144a36bdcff69ccd5d493b P c931513400d34c2ea21dd0b799184d12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:02:23.152889 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:23.152598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:59085)
0810 21:02:23.152760 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:23.152841 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260810 21:02:23.196957 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:23.196572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:37145)
0810 21:02:23.196880 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:23.196916 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":247,"threads_started":1}
I20260810 21:02:23.327364 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:23.327191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:60439)
0810 21:02:23.327299 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:23.327328 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260810 21:02:24.153652 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:24.153439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:56645)
0810 21:02:24.153567 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:24.153606 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260810 21:02:24.197677 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:24.197424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:45553)
0810 21:02:24.197598 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:24.197634 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260810 21:02:24.328013 31896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:24.327787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:53503)
0810 21:02:24.327946 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:24.327975 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":112,"threads_started":1}
I20260810 21:02:25.154412 31897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:25.154136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:49839)
0810 21:02:25.154306 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:25.154342 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
I20260810 21:02:25.198411 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:25.198210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:33103)
0810 21:02:25.198331 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:25.198367 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260810 21:02:25.328647 31899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:25.328454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:39143)
0810 21:02:25.328579 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:25.328610 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260810 21:02:26.155189 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:26.154946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:49195)
0810 21:02:26.155104 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:26.155141 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260810 21:02:26.199009 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:26.198819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:54137)
0810 21:02:26.198943 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:26.198975 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260810 21:02:26.329298 31902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:26.329094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:60239)
0810 21:02:26.329221 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:26.329256 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260810 21:02:27.156066 31903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:27.155789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.1:58939)
0810 21:02:27.155941 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:27.156000 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260810 21:02:27.199633 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:27.199451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.2:38297)
0810 21:02:27.199558 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:27.199591 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260810 21:02:27.329909 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:27.329733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36959 (local address 127.14.200.3:44101)
0810 21:02:27.329842 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:27.329872 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260810 21:02:27.333227 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 31374
I20260810 21:02:27.344990 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 31553
I20260810 21:02:27.352370 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 31697
2026-08-10T21:02:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18370 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:02:27 dist-test-slave-1x4g krb5kdc[31908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:02:27 dist-test-slave-1x4g krb5kdc[31908](info): set up 2 sockets
Aug 10 21:02:27 dist-test-slave-1x4g krb5kdc[31908](info): commencing operation
krb5kdc: starting...
W20260810 21:02:28.600649 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:02:28 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:02:28.805117 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:28.978005 31922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:28.978338 31922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:28.978371 31922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:28.980036 31922 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 21:02:28.980095 31922 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 21:02:28.980120 31922 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 21:02:28.980144 31922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:28.980175 31922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:28.980207 31922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:28.980237 31922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:28.980264 31922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:28.982627 31922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:28.983063 31922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:28.983366 31922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:28.991705 31931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:28.991660 31929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:28.991730 31928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:28.991683 31922 server_base.cc:1034] running on GCE node
I20260810 21:02:28.992462 31922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:28.993050 31922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:28.994287 31922 hybrid_clock.cc:648] HybridClock initialized: now 1786395748994261 us; error 80 us; skew 500 ppm
Aug 10 21:02:28 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395748, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:28.998555 31922 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:28.999400 31922 webserver.cc:460] Webserver started at http://127.14.200.62:39211/ using document root <none> and password file <none>
I20260810 21:02:28.999778 31922 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:28.999912 31922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:29.000144 31922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:29.001976 31922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "f8710528382c47aba5afcf09d7b5bf5e"
format_stamp: "Formatted at 2026-08-10 21:02:28 on dist-test-slave-1x4g"
server_key: "15ebc9b35ced6b77f4f2e78c272558cd"
server_key_iv: "2569b6281be0fe757286405099d66f35"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.002622 31922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "f8710528382c47aba5afcf09d7b5bf5e"
format_stamp: "Formatted at 2026-08-10 21:02:28 on dist-test-slave-1x4g"
server_key: "15ebc9b35ced6b77f4f2e78c272558cd"
server_key_iv: "2569b6281be0fe757286405099d66f35"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.005168 31922 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260810 21:02:29.006516 31939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.006898 31922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:29.007074 31922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f8710528382c47aba5afcf09d7b5bf5e"
format_stamp: "Formatted at 2026-08-10 21:02:28 on dist-test-slave-1x4g"
server_key: "15ebc9b35ced6b77f4f2e78c272558cd"
server_key_iv: "2569b6281be0fe757286405099d66f35"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.007274 31922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:29.039871 31922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:29.043901 31922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:29.044137 31922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:29.053305 31922 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36683
I20260810 21:02:29.053427 31997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36683 every 8 connection(s)
I20260810 21:02:29.054183 31922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:29.056308 31999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:29.057813 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 31922
I20260810 21:02:29.057931 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:02:29.058408 15136 external_mini_cluster.cc:1428] Setting key 3fc1e39976c7415dded8cda60d0f72e7
I20260810 21:02:29.064615 31999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap starting.
I20260810 21:02:29.069278 31999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:29.070096 31999 log.cc:824] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:29.071790 31999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: No bootstrap required, opened a new log
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:29.078174 31999 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:29.078636 31999 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:29.078751 31999 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Initialized, Role: FOLLOWER
I20260810 21:02:29.079138 31999 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:29.079344 31999 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:29.079413 31999 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:29.079568 31999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:29.080632 31999 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:29.080891 31999 leader_election.cc:302] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8710528382c47aba5afcf09d7b5bf5e; no voters:
I20260810 21:02:29.081208 31999 leader_election.cc:288] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:02:29.081272 32005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:29.081456 32005 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 LEADER]: Becoming Leader. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Running, Role: LEADER
I20260810 21:02:29.081807 31999 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:29.081761 32005 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:29.083274 32006 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:02:29.083491 32006 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:29.084232 32007 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8710528382c47aba5afcf09d7b5bf5e. Latest consensus state: current_term: 1 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:02:29.084436 32007 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
W20260810 21:02:29.086009 32027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:02:29.086236 32027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:02:29.086508 32017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:29.086927 32017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:29.087379 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.059068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52478 (local address 127.14.200.62:36683)
0810 21:02:29.059371 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.059378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.059405 (+ 27us) server_negotiation.cc:407] Connection header received
0810 21:02:29.060634 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.060719 (+ 85us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.061088 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.061458 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:29.064404 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.069383 (+ 4979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.071561 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.072380 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.072530 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.079072 (+ 6542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:29.079084 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:29.079084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:29.079105 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:29.082947 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.083865 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.083869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.083870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.084081 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.084636 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.084636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.084637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.085782 (+ 1145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:29.086176 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.086485 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.086863 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":120,"threads_started":1}
I20260810 21:02:29.089931 32017 catalog_manager.cc:1347] Generated new cluster ID: c60b2e52db82433c93745a6fd47830e6
I20260810 21:02:29.090042 32017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:02:29.113150 32017 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:02:29.114683 32017 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:29.131801 32017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Generated new TSK 0
I20260810 21:02:29.132177 32017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:02:29.185995 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:29.363693 32033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:29.364133 32033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:29.364295 32033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:29.366464 32033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:29.366566 32033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:29.366711 32033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:29.369315 32033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:29.369812 32033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:29.370278 32033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:29.379009 32039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:29.379014 32042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:29.379040 32038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:29.379678 32033 server_base.cc:1034] running on GCE node
I20260810 21:02:29.380126 32033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:29.380604 32033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:29.381887 32033 hybrid_clock.cc:648] HybridClock initialized: now 1786395749381797 us; error 65 us; skew 500 ppm
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:29.386561 32033 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:29.387087 32033 webserver.cc:460] Webserver started at http://127.14.200.1:45513/ using document root <none> and password file <none>
I20260810 21:02:29.387400 32033 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:29.387526 32033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:29.387774 32033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:29.389876 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "035b3a10aaec457cb171273b2922da75"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "f0c420fc6243c6f52eec73c254a29a42"
server_key_iv: "df8636d19a4f543b22c3bf7dde93ef64"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.390573 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "035b3a10aaec457cb171273b2922da75"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "f0c420fc6243c6f52eec73c254a29a42"
server_key_iv: "df8636d19a4f543b22c3bf7dde93ef64"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.393347 32033 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260810 21:02:29.394640 32049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.394913 32033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:29.395092 32033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "035b3a10aaec457cb171273b2922da75"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "f0c420fc6243c6f52eec73c254a29a42"
server_key_iv: "df8636d19a4f543b22c3bf7dde93ef64"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.395200 32033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:29.410234 32033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:29.413934 32033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:29.414233 32033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:29.414793 32033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:29.415380 32033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:29.415488 32033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.415622 32033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:29.415727 32033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.429440 32033 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46881
I20260810 21:02:29.429508 32176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46881 every 8 connection(s)
I20260810 21:02:29.430076 32033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:29.438162 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 32033
I20260810 21:02:29.438409 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:02:29.439095 15136 external_mini_cluster.cc:1428] Setting key daee0ad64869ecdf04c659e87e88b068
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:29.446872 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.431209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:46103 (local address 127.14.200.62:36683)
0810 21:02:29.431326 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.431329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.431725 (+ 396us) server_negotiation.cc:407] Connection header received
0810 21:02:29.432609 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.432611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.432632 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.432739 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:29.434544 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.435327 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.436752 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.437173 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.437262 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.441778 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:29.441779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:29.441780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:29.441787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:29.444418 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.445190 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.445191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.445191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.445278 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.445723 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.445724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.445724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.445895 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:29.446110 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.446564 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.446710 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 21:02:29.447244 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.431204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:46103)
0810 21:02:29.431534 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.431608 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.432417 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.432961 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.432964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.434066 (+ 1102us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:29.434400 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.434405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.435466 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.435467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.436662 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.436664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.437301 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.437303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.437467 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.438409 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:29.438431 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:29.441676 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:29.444547 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.444563 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.444567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.445028 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.445390 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.445390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.445391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.445564 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.446084 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:29.446091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:29.446304 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.446499 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.446793 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":210,"threads_started":1}
I20260810 21:02:29.448093 32177 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:29.448212 32177 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:29.448544 32177 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:29.449434 31959 ts_manager.cc:194] Registered new tserver with Master: 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:29.450291 31959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:46103
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:02:29.480014 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:29.647075 32182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:29.647475 32182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:29.647527 32182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:29.649672 32182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:29.649753 32182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:29.649801 32182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:29.652520 32182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:29.652940 32182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:29.653277 32182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:29.662969 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:29.663111 32189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:29.663586 32182 server_base.cc:1034] running on GCE node
W20260810 21:02:29.663666 32191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:29.664075 32182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:29.664532 32182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:29.665781 32182 hybrid_clock.cc:648] HybridClock initialized: now 1786395749665744 us; error 68 us; skew 500 ppm
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:29.670553 32182 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:29.671479 32182 webserver.cc:460] Webserver started at http://127.14.200.2:44541/ using document root <none> and password file <none>
I20260810 21:02:29.671936 32182 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:29.672063 32182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:29.672271 32182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:29.674049 32182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "20ffc10c8ff144e888351475ba25e149"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "c554f59399618c77e8c1a4005ce47f70"
server_key_iv: "38f1a33e69ce301db735e8e0f645ddb7"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.674834 32182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "20ffc10c8ff144e888351475ba25e149"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "c554f59399618c77e8c1a4005ce47f70"
server_key_iv: "38f1a33e69ce301db735e8e0f645ddb7"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.677608 32182 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 21:02:29.678930 32199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.679245 32182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:29.679397 32182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "20ffc10c8ff144e888351475ba25e149"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "c554f59399618c77e8c1a4005ce47f70"
server_key_iv: "38f1a33e69ce301db735e8e0f645ddb7"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.679504 32182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:29.733525 32182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:29.737142 32182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:29.737358 32182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:29.737682 32182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:29.738090 32182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:29.738137 32182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.738179 32182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:29.738221 32182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.748554 32182 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:40481
I20260810 21:02:29.748656 32322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:40481 every 8 connection(s)
I20260810 21:02:29.749243 32182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:02:29.749866 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 32182
I20260810 21:02:29.750072 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:02:29.750418 15136 external_mini_cluster.cc:1428] Setting key ef7edfb9b34ba65dc2eb8e2a76ce555a
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
I20260810 21:02:29.766390 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.751674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:57707 (local address 127.14.200.62:36683)
0810 21:02:29.751744 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.751746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.752159 (+ 413us) server_negotiation.cc:407] Connection header received
0810 21:02:29.753543 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.753544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.753563 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.753625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:29.755843 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.756549 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.758163 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.758435 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.758591 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.761920 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:29.761922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:29.761922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:29.761933 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:29.764068 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.764754 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.764755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.764755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.764910 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.765338 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.765339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.765339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.765509 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:29.765692 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.766189 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.766294 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 21:02:29.766654 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.751601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:57707)
0810 21:02:29.751884 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.751976 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.753384 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.753789 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.753803 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.755308 (+ 1505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:29.755656 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.755668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.756732 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.756733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.758006 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.758009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.758595 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.758597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.758799 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.759618 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:29.759628 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:29.761738 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:29.764223 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.764247 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.764251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.764604 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.765034 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.765035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.765035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.765207 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.765677 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:29.765692 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:29.765884 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.766120 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.766339 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":144,"threads_started":1}
I20260810 21:02:29.767480 32323 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:29.767649 32323 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
I20260810 21:02:29.767982 32323 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:29.768566 31959 ts_manager.cc:194] Registered new tserver with Master: 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
I20260810 21:02:29.769129 31959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:57707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:02:29.787917 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:29.919641 32329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:29.919857 32329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:29.919893 32329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:29.921056 32329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:29.921109 32329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:29.921149 32329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:29.922547 32329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:29.922816 32329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:29.923041 32329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:29.927908 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:29.927913 32329 server_base.cc:1034] running on GCE node
W20260810 21:02:29.927974 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:29.928093 32338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:29.928349 32329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:29.928547 32329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:29.929678 32329 hybrid_clock.cc:648] HybridClock initialized: now 1786395749929669 us; error 25 us; skew 500 ppm
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:29.932197 32329 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:29.932623 32329 webserver.cc:460] Webserver started at http://127.14.200.3:34843/ using document root <none> and password file <none>
I20260810 21:02:29.932891 32329 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:29.932969 32329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:29.933115 32329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:02:29.934079 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "fc0f62e39fa249e88c46177d2a24ce93"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "45bd3cc4a037b2c2c7305d50e3c72aea"
server_key_iv: "d3073072d220281913fe00b4f738ca00"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.934413 32329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "fc0f62e39fa249e88c46177d2a24ce93"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "45bd3cc4a037b2c2c7305d50e3c72aea"
server_key_iv: "d3073072d220281913fe00b4f738ca00"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.935786 32329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:02:29.936471 32346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.936637 32329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:02:29.936760 32329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "fc0f62e39fa249e88c46177d2a24ce93"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "45bd3cc4a037b2c2c7305d50e3c72aea"
server_key_iv: "d3073072d220281913fe00b4f738ca00"
server_key_version: "encryptionkey@0"
I20260810 21:02:29.936869 32329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:29.953593 32329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:29.955721 32329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:29.955875 32329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:29.956089 32329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:29.956389 32329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:29.956437 32329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.956468 32329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:29.956494 32329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:29.962218 32329 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:36027
I20260810 21:02:29.962265 32470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:36027 every 8 connection(s)
I20260810 21:02:29.962536 32329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:29.964324 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 32329
I20260810 21:02:29.964406 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:02:29.964617 15136 external_mini_cluster.cc:1428] Setting key 6f9716ee8a1d98e8ed1a777ac9ed00c0
Aug 10 21:02:29 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395749, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:29.971268 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.963205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:52985 (local address 127.14.200.62:36683)
0810 21:02:29.963259 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.963261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.963415 (+ 154us) server_negotiation.cc:407] Connection header received
0810 21:02:29.963883 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.963884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.963895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.963929 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:29.965077 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.965796 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.966737 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.966917 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.966970 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.968889 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:29.968890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:29.968891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:29.968897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:29.970241 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.970563 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.970564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.970564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.970610 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.970791 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.970792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.970792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.970877 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:29.970952 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.971152 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.971208 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 21:02:29.971410 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.963166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:52985)
0810 21:02:29.963313 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.963343 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.963796 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.963986 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.963989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.964663 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:29.964973 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.964981 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.965875 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.965876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.966658 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.966660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.966989 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.966989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.967097 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.967487 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:29.967494 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:29.968774 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:29.970304 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.970309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.970310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.970508 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.970660 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:29.970660 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:29.970660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:29.970746 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:29.970943 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:29.970947 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:29.971054 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.971128 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.971234 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260810 21:02:29.971825 32471 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:29.971907 32471 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:29.972052 32471 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:29.972391 31959 ts_manager.cc:194] Registered new tserver with Master: fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027)
I20260810 21:02:29.972746 31959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:52985
I20260810 21:02:29.976198 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:29.982833 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.977554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.14.200.62:36683)
0810 21:02:29.977618 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.977620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.977705 (+ 85us) server_negotiation.cc:407] Connection header received
0810 21:02:29.977774 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.977775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.977782 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.977816 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:29.978290 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.978690 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.979323 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.979484 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.979528 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.980482 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:29.980483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:29.980483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:29.980488 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:29.981782 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.982150 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.982151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.982151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.982200 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:29.982342 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:29.982343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:29.982343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:29.982467 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:29.982543 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.982632 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.982710 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 21:02:29.984359 31959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:02:29.986063 31959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:02:29.992095 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:36027 (local address 127.0.0.1:59052)
0810 21:02:29.988330 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.988370 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.988433 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.988965 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.988967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.988969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:29.989033 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.989035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.989900 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.989901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.991001 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.991004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.991801 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.991802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.991886 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.991899 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.991972 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.992021 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260810 21:02:29.992121 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.14.200.3:36027)
0810 21:02:29.988720 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.988725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.988735 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:29.988773 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.988775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.988808 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.989114 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:29.989172 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.989806 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.991072 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.991718 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.991792 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.991843 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.991997 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.992043 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":82,"threads_started":1}
I20260810 21:02:29.992544 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40481 (local address 127.0.0.1:58374)
0810 21:02:29.988518 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.988526 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.988570 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.988939 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.988942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.988956 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:29.989055 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.989057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.990345 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.990346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.991277 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.991279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.992277 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.992278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.992348 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.992351 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.992417 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.992457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":6,"thread_start_us":78,"threads_started":1}
I20260810 21:02:29.992552 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.14.200.2:40481)
0810 21:02:29.988740 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.988744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.988753 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:02:29.988806 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.988808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.988822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.989144 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:29.989231 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.990258 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.991387 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.992114 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.992183 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.992242 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.992433 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.992471 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":106,"threads_started":1}
I20260810 21:02:29.992853 32398 tablet_service.cc:1467] Processing CreateTablet for tablet bbc7717910f24a21a426bc53794f76ae (DEFAULT_TABLE table=test-table [id=d9302c86f7024185b605c673feda1abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:29.993212 32398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc7717910f24a21a426bc53794f76ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:29.993323 32249 tablet_service.cc:1467] Processing CreateTablet for tablet bbc7717910f24a21a426bc53794f76ae (DEFAULT_TABLE table=test-table [id=d9302c86f7024185b605c673feda1abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:29.993650 32249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc7717910f24a21a426bc53794f76ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:29.994154 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.0.0.1:58476)
0810 21:02:29.988631 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:29.988657 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:29.988712 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:29.988968 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:29.988968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:29.988970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:29.989067 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.989069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.991383 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.991384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.992906 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:29.992909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.993928 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.993929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:29.994002 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.994005 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:02:29.994055 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 21:02:29.994090 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":29,"thread_start_us":85,"threads_started":1}
I20260810 21:02:29.994421 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:29.988489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.14.200.1:46881)
0810 21:02:29.988654 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:29.988659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:29.988722 (+ 63us) server_negotiation.cc:407] Connection header received
0810 21:02:29.988795 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:29.988797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:29.988815 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:29.988899 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:29.989152 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.990259 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.993051 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:29.993783 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:29.993850 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:29.993896 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:29.994301 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 21:02:29.994348 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260810 21:02:29.994987 32102 tablet_service.cc:1467] Processing CreateTablet for tablet bbc7717910f24a21a426bc53794f76ae (DEFAULT_TABLE table=test-table [id=d9302c86f7024185b605c673feda1abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:29.995303 32102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbc7717910f24a21a426bc53794f76ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:29.997005 32492 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Bootstrap starting.
I20260810 21:02:29.997661 32492 tablet_bootstrap.cc:654] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:29.998238 32492 log.cc:824] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:29.998571 32491 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Bootstrap starting.
I20260810 21:02:29.998764 32494 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Bootstrap starting.
I20260810 21:02:29.998996 32492 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: No bootstrap required, opened a new log
I20260810 21:02:29.999065 32492 ts_tablet_manager.cc:1397] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:02:29.999163 32491 tablet_bootstrap.cc:654] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:29.999368 32494 tablet_bootstrap.cc:654] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:29.999660 32491 log.cc:824] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:29.999691 32494 log.cc:824] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:30.000639 32492 raft_consensus.cc:348] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.000861 32492 raft_consensus.cc:374] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:30.000926 32492 raft_consensus.cc:729] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0f62e39fa249e88c46177d2a24ce93, State: Initialized, Role: FOLLOWER
I20260810 21:02:30.001117 32492 consensus_queue.cc:260] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.001379 32491 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: No bootstrap required, opened a new log
I20260810 21:02:30.001425 32492 ts_tablet_manager.cc:1428] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:02:30.001462 32491 ts_tablet_manager.cc:1397] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 21:02:30.001544 32471 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
I20260810 21:02:30.001595 32494 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: No bootstrap required, opened a new log
I20260810 21:02:30.001668 32494 ts_tablet_manager.cc:1397] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 21:02:30.003234 32491 raft_consensus.cc:348] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.003374 32491 raft_consensus.cc:374] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:30.003413 32491 raft_consensus.cc:729] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ffc10c8ff144e888351475ba25e149, State: Initialized, Role: FOLLOWER
I20260810 21:02:30.003563 32491 consensus_queue.cc:260] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.003528 32494 raft_consensus.cc:348] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.003664 32494 raft_consensus.cc:374] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:30.003690 32494 raft_consensus.cc:729] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b3a10aaec457cb171273b2922da75, State: Initialized, Role: FOLLOWER
I20260810 21:02:30.003785 32494 consensus_queue.cc:260] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.003839 32491 ts_tablet_manager.cc:1428] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:02:30.003942 32323 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
I20260810 21:02:30.004025 32494 ts_tablet_manager.cc:1428] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:02:30.004094 32177 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
W20260810 21:02:30.181313 32178 tablet.cc:2369] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:30.213097 32472 tablet.cc:2369] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:30.250192 32324 tablet.cc:2369] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:30.306625 32498 raft_consensus.cc:482] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:30.306754 32498 raft_consensus.cc:504] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.307174 32498 leader_election.cc:288] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:30.310114 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.307507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50709 (local address 127.14.200.3:36027)
0810 21:02:30.307584 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:30.307586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:30.307596 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:30.307730 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:30.307731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:30.307743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:30.307798 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:30.307974 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.308440 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309389 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.309770 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309825 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:30.309832 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:30.309993 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 21:02:30.310041 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 21:02:30.310129 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.307379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:36027 (local address 127.14.200.2:50709)
0810 21:02:30.307517 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:30.307528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:30.307611 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:30.307813 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:30.307814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:30.307817 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:30.307902 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:30.307905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.308517 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.308518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:30.309313 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:30.309315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.309848 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:30.309918 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:30.309927 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 21:02:30.309974 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 21:02:30.310019 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":11,"thread_start_us":89,"threads_started":1}
I20260810 21:02:30.310141 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.307662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:55337 (local address 127.14.200.1:46881)
0810 21:02:30.307722 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:30.307724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:30.307731 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:30.307768 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:30.307768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:30.307787 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:30.307842 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:30.308028 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.308464 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309455 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.309816 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309886 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:30.309894 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:30.310040 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 21:02:30.310077 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:02:30.310169 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.307574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.14.200.2:55337)
0810 21:02:30.307661 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:30.307670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:30.307721 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:30.307856 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:30.307857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:30.307859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:30.307947 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:30.307950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.308557 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.308558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:30.309341 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:30.309343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.309900 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.309901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:30.309952 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:30.309956 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:30.310032 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 21:02:30.310068 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":2,"thread_start_us":39,"threads_started":1}
I20260810 21:02:30.310465 32418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc7717910f24a21a426bc53794f76ae" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0f62e39fa249e88c46177d2a24ce93" is_pre_election: true
I20260810 21:02:30.310633 32418 raft_consensus.cc:2454] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 0.
I20260810 21:02:30.310717 32124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc7717910f24a21a426bc53794f76ae" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b3a10aaec457cb171273b2922da75" is_pre_election: true
I20260810 21:02:30.310887 32124 raft_consensus.cc:2454] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 0.
I20260810 21:02:30.310899 32200 leader_election.cc:302] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93; no voters:
I20260810 21:02:30.311023 32498 raft_consensus.cc:2781] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:30.311072 32498 raft_consensus.cc:482] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:30.311087 32498 raft_consensus.cc:3037] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:30.311798 32498 raft_consensus.cc:504] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.311919 32498 leader_election.cc:288] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 election: Requested vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:30.312088 32418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc7717910f24a21a426bc53794f76ae" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0f62e39fa249e88c46177d2a24ce93"
I20260810 21:02:30.312115 32124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbc7717910f24a21a426bc53794f76ae" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b3a10aaec457cb171273b2922da75"
I20260810 21:02:30.312182 32418 raft_consensus.cc:3037] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:30.312180 32124 raft_consensus.cc:3037] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:30.312906 32124 raft_consensus.cc:2454] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 1.
I20260810 21:02:30.312975 32418 raft_consensus.cc:2454] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 1.
I20260810 21:02:30.313072 32200 leader_election.cc:302] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035b3a10aaec457cb171273b2922da75, 20ffc10c8ff144e888351475ba25e149; no voters:
I20260810 21:02:30.313143 32498 raft_consensus.cc:2781] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:30.313293 32498 raft_consensus.cc:686] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 LEADER]: Becoming Leader. State: Replica: 20ffc10c8ff144e888351475ba25e149, State: Running, Role: LEADER
I20260810 21:02:30.313387 32498 consensus_queue.cc:237] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:30.314220 31958 catalog_manager.cc:5495] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ffc10c8ff144e888351475ba25e149 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "20ffc10c8ff144e888351475ba25e149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } health_report { overall_health: HEALTHY } } }
I20260810 21:02:30.343088 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.340942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.14.200.2:40481)
0810 21:02:30.340997 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:30.340999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:30.341006 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:30.341081 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:30.341082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:30.341098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:30.341166 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:30.341267 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.341705 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.342434 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:30.342611 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:30.342659 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:30.342796 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:30.342885 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:30.342941 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:30.342978 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 21:02:30.342993 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 21:02:30.344862 32124 raft_consensus.cc:1261] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Refusing update from remote peer 20ffc10c8ff144e888351475ba25e149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:30.344921 32418 raft_consensus.cc:1261] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Refusing update from remote peer 20ffc10c8ff144e888351475ba25e149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:30.345263 32498 consensus_queue.cc:1035] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:30.345402 32503 consensus_queue.cc:1035] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:30.348973 32508 mvcc.cc:205] Tried to move back new op lower bound from 7317076993409732608 to 7317076993284165632. Current Snapshot: MvccSnapshot[applied={T|T < 7317076993409732608}]
I20260810 21:02:30.350169 31958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:30.350252 31958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:30.350945 31958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Sending DeleteTablet for 3 replicas of tablet bbc7717910f24a21a426bc53794f76ae
I20260810 21:02:30.351239 32398 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc7717910f24a21a426bc53794f76ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:59052
I20260810 21:02:30.351296 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 32033
I20260810 21:02:30.351454 32249 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc7717910f24a21a426bc53794f76ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:58374
I20260810 21:02:30.353243 32517 tablet_replica.cc:330] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: stopping tablet replica
I20260810 21:02:30.353344 32517 raft_consensus.cc:2229] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:30.353427 32517 raft_consensus.cc:2258] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:30.353919 32517 ts_tablet_manager.cc:1905] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:30.355901 32517 ts_tablet_manager.cc:1918] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:30.355978 32517 log.cc:1196] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/bbc7717910f24a21a426bc53794f76ae
I20260810 21:02:30.356285 32517 ts_tablet_manager.cc:1939] T bbc7717910f24a21a426bc53794f76ae P fc0f62e39fa249e88c46177d2a24ce93: Deleting consensus metadata
I20260810 21:02:30.356729 31940 catalog_manager.cc:4841] TS fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): tablet bbc7717910f24a21a426bc53794f76ae (table test-table [id=d9302c86f7024185b605c673feda1abb]) successfully deleted
W20260810 21:02:30.359929 31940 connection.cc:535] client connection to 127.14.200.1:46881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 21:02:30.360038 31940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:02:30.360239 32518 tablet_replica.cc:330] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: stopping tablet replica
I20260810 21:02:30.360345 32518 raft_consensus.cc:2229] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:02:30.360457 32518 raft_consensus.cc:2258] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:30.360500 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.360378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.0.0.1:58482)
0810 21:02:30.360428 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:30.360463 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 21:02:30.360626 31940 catalog_manager.cc:4584] TS 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbc7717910f24a21a426bc53794f76ae: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46881: connect: Connection refused (error 111)
I20260810 21:02:30.360669 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 32182
I20260810 21:02:30.369338 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 32329
I20260810 21:02:30.369382 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:30.369261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40481 (local address 127.0.0.1:58388)
0810 21:02:30.369316 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:30.369348 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.2:40481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 21:02:30.376514 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 31922
I20260810 21:02:50.383328 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--webserver_interface=127.14.200.62
--webserver_port=39211
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:02:50.464286 32521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:50.464493 32521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:50.464524 32521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:50.465601 32521 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 21:02:50.465636 32521 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 21:02:50.465649 32521 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 21:02:50.465660 32521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:02:50.465710 32521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:50.465727 32521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:50.465749 32521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:02:50.465772 32521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:02:50.467245 32521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=39211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:50.467485 32521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:50.467736 32521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:50.477094 32532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.477103 32527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.477185 32528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:50.477324 32521 server_base.cc:1034] running on GCE node
I20260810 21:02:50.477690 32521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:50.478071 32521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:50.479241 32521 hybrid_clock.cc:648] HybridClock initialized: now 1786395770479230 us; error 46 us; skew 500 ppm
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:50.486541 32521 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:02:50.487010 32521 webserver.cc:460] Webserver started at http://127.14.200.62:39211/ using document root <none> and password file <none>
I20260810 21:02:50.487226 32521 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:50.487280 32521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:50.488616 32521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:50.489288 32540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.489487 32521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.489583 32521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f8710528382c47aba5afcf09d7b5bf5e"
format_stamp: "Formatted at 2026-08-10 21:02:28 on dist-test-slave-1x4g"
server_key: "15ebc9b35ced6b77f4f2e78c272558cd"
server_key_iv: "2569b6281be0fe757286405099d66f35"
server_key_version: "encryptionkey@0"
I20260810 21:02:50.489987 32521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:50.521690 32521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:50.523610 32521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:50.523767 32521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:50.528106 32521 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36683
I20260810 21:02:50.528170 32598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36683 every 8 connection(s)
I20260810 21:02:50.528630 32521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:02:50.530062 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 32521
I20260810 21:02:50.530294 32599 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:02:50.530380 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:46881
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=45513
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:50.531069 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap starting.
I20260810 21:02:50.541651 32599 log.cc:824] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:50.544405 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:50.544796 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap complete.
I20260810 21:02:50.548421 32599 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:50.548707 32599 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Initialized, Role: FOLLOWER
I20260810 21:02:50.548870 32599 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:50.548936 32599 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:02:50.548995 32599 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:02:50.549041 32599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 1 FOLLOWER]: Advancing to term 2
I20260810 21:02:50.550309 32599 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:50.550431 32599 leader_election.cc:302] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8710528382c47aba5afcf09d7b5bf5e; no voters:
I20260810 21:02:50.550580 32599 leader_election.cc:288] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 21:02:50.550642 32606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Leader election won for term 2
I20260810 21:02:50.550810 32599 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:02:50.550837 32606 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 LEADER]: Becoming Leader. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Running, Role: LEADER
I20260810 21:02:50.551074 32606 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:02:50.551424 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:02:50.551505 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:50.552088 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8710528382c47aba5afcf09d7b5bf5e. Latest consensus state: current_term: 2 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:02:50.552138 32615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:02:50.552178 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
I20260810 21:02:50.552927 32615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:02:50.553035 32615 catalog_manager.cc:1259] Loaded cluster ID: c60b2e52db82433c93745a6fd47830e6
I20260810 21:02:50.553073 32615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:02:50.553948 32615 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:02:50.554157 32615 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Loaded TSK: 0
I20260810 21:02:50.567556 32615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Generated new TSK 1
I20260810 21:02:50.567668 32615 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:02:50.640477 32601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:50.640662 32601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:50.640684 32601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:50.641898 32601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:50.641952 32601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:50.641991 32601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:02:50.643381 32601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:46881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=45513
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:50.643606 32601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:50.643859 32601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:50.648741 32634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:50.648979 32601 server_base.cc:1034] running on GCE node
W20260810 21:02:50.648757 32636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.648761 32633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:50.649294 32601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:50.649580 32601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:50.650756 32601 hybrid_clock.cc:648] HybridClock initialized: now 1786395770650749 us; error 33 us; skew 500 ppm
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:50.653959 32601 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:02:50.654450 32601 webserver.cc:460] Webserver started at http://127.14.200.1:45513/ using document root <none> and password file <none>
I20260810 21:02:50.654716 32601 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:50.654796 32601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:50.656407 32601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:02:50.657253 32644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.657473 32601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:50.657582 32601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "035b3a10aaec457cb171273b2922da75"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "f0c420fc6243c6f52eec73c254a29a42"
server_key_iv: "df8636d19a4f543b22c3bf7dde93ef64"
server_key_version: "encryptionkey@0"
I20260810 21:02:50.657999 32601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:50.673379 32601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:50.676144 32601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:50.676290 32601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:50.676545 32601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:50.678342 32651 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 21:02:50.679834 32601 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 21:02:50.679894 32601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.001s
I20260810 21:02:50.679929 32601 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 21:02:50.680873 32601 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 21:02:50.680917 32601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260810 21:02:50.681598 32651 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Bootstrap starting.
I20260810 21:02:50.697923 32601 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46881
I20260810 21:02:50.697975 300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46881 every 8 connection(s)
I20260810 21:02:50.698477 32601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:02:50.699712 32651 log.cc:824] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:50.701275 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 32601
I20260810 21:02:50.701617 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:40481
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=44541
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:50.705610 32651 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:50.705955 32651 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Bootstrap complete.
I20260810 21:02:50.707229 32651 ts_tablet_manager.cc:1397] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.012s
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:50.708541 32651 raft_consensus.cc:348] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:50.708902 32651 raft_consensus.cc:729] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b3a10aaec457cb171273b2922da75, State: Initialized, Role: FOLLOWER
I20260810 21:02:50.709070 32651 consensus_queue.cc:260] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:50.709353 32651 ts_tablet_manager.cc:1428] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260810 21:02:50.713804 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.699320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:41113)
0810 21:02:50.699768 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:50.699811 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:50.700354 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:50.701012 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:50.701016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:50.701501 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:50.701998 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.702005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.704549 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.704551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.705760 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.705763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.706577 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.706579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.706725 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.707348 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:50.707359 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:50.709328 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:50.711813 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.711822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.711824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.712156 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.712462 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.712463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.712464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.712608 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.713113 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:50.713121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:50.713311 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 21:02:50.713441 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 21:02:50.713611 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":84,"threads_started":1}
I20260810 21:02:50.714747 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.699584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:41113 (local address 127.14.200.62:36683)
0810 21:02:50.700545 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.700554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.700587 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:02:50.700710 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.700715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.700761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.700914 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.702136 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.704398 (+ 2262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.705847 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.706453 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.706557 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.709499 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:50.709504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:50.709504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:50.709528 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:50.711713 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.712255 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.712258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.712259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.712383 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.712678 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.712679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.712679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.713021 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:50.713132 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.713541 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.713699 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":82,"threads_started":1}
I20260810 21:02:50.715646 301 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:50.715751 301 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:50.715957 301 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:50.716775 32558 ts_manager.cc:194] Registered new tserver with Master: 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:50.717888 32558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:41113
I20260810 21:02:50.719116 301 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
I20260810 21:02:50.724332 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.717465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.0.0.1:46632)
0810 21:02:50.719643 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:50.719680 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:50.719756 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:50.720008 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:50.720012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:50.720014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:50.720083 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.720085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.721484 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.721486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.722925 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.722928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.724009 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.724011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.724105 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.724118 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 21:02:50.724184 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 21:02:50.724240 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2148,"thread_start_us":85,"threads_started":1}
I20260810 21:02:50.724650 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.718224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46632 (local address 127.14.200.1:46881)
0810 21:02:50.718389 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.718394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.719739 (+ 1345us) server_negotiation.cc:407] Connection header received
0810 21:02:50.719842 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.719844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.719864 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.719939 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:50.720159 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.721376 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.723029 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.723830 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.723905 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.723971 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.724526 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.724574 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260810 21:02:50.724999 32697 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc7717910f24a21a426bc53794f76ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:46632
I20260810 21:02:50.725623 310 tablet_replica.cc:330] stopping tablet replica
I20260810 21:02:50.725790 310 raft_consensus.cc:2229] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:50.725908 310 raft_consensus.cc:2258] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:50.726346 310 ts_tablet_manager.cc:1905] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:50.729270 310 ts_tablet_manager.cc:1918] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:50.729382 310 log.cc:1196] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal/wals/bbc7717910f24a21a426bc53794f76ae
I20260810 21:02:50.729686 310 ts_tablet_manager.cc:1939] T bbc7717910f24a21a426bc53794f76ae P 035b3a10aaec457cb171273b2922da75: Deleting consensus metadata
I20260810 21:02:50.730257 32541 catalog_manager.cc:4841] TS 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881): tablet bbc7717910f24a21a426bc53794f76ae (table test-table [id=d9302c86f7024185b605c673feda1abb]) successfully deleted
W20260810 21:02:50.799856 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:50.800076 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:50.800112 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:50.801246 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:50.801303 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:50.801347 306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:02:50.802690 306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:40481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=44541
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:50.802937 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:50.803150 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:50.807791 319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.807785 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.807765 315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:50.807948 306 server_base.cc:1034] running on GCE node
I20260810 21:02:50.808182 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:50.808398 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:50.809547 306 hybrid_clock.cc:648] HybridClock initialized: now 1786395770809524 us; error 40 us; skew 500 ppm
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:50.811954 306 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:02:50.812312 306 webserver.cc:460] Webserver started at http://127.14.200.2:44541/ using document root <none> and password file <none>
I20260810 21:02:50.812503 306 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:50.812551 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:50.813772 306 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:50.814339 326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.814508 306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:02:50.814611 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "20ffc10c8ff144e888351475ba25e149"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "c554f59399618c77e8c1a4005ce47f70"
server_key_iv: "38f1a33e69ce301db735e8e0f645ddb7"
server_key_version: "encryptionkey@0"
I20260810 21:02:50.814944 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:50.838896 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:50.840674 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:50.840852 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:50.841106 306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:50.841532 333 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 21:02:50.843027 306 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 21:02:50.843086 306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 21:02:50.843128 306 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 21:02:50.843765 306 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 21:02:50.843801 306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:50.843886 333 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Bootstrap starting.
I20260810 21:02:50.850847 306 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:40481
I20260810 21:02:50.850936 451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:40481 every 8 connection(s)
I20260810 21:02:50.851281 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:02:50.854614 333 log.cc:824] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:50.857210 333 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:02:50.857559 333 tablet_bootstrap.cc:492] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Bootstrap complete.
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:50.858678 333 ts_tablet_manager.cc:1397] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260810 21:02:50.859544 333 raft_consensus.cc:348] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:50.859781 333 raft_consensus.cc:729] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ffc10c8ff144e888351475ba25e149, State: Initialized, Role: FOLLOWER
I20260810 21:02:50.859936 333 consensus_queue.cc:260] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:50.860194 333 ts_tablet_manager.cc:1428] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260810 21:02:50.860702 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 306
I20260810 21:02:50.861019 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:36027
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=34843
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:02:50.861733 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.852071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:58007 (local address 127.14.200.62:36683)
0810 21:02:50.852133 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.852135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.852396 (+ 261us) server_negotiation.cc:407] Connection header received
0810 21:02:50.853172 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.853173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.853184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.853227 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.854146 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.854620 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.856010 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.856206 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.856262 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.858766 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:50.858768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:50.858768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:50.858777 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:50.860309 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.860700 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.860701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.860701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.860760 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.861031 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.861032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.861033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.861185 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:50.861247 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.861595 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.861674 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 21:02:50.861940 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.852021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:58007)
0810 21:02:50.852240 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:50.852292 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:50.853063 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:50.853298 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:50.853301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:50.853711 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:50.854052 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.854061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.854776 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.854778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.855920 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.855923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.856305 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.856306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.856456 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.857052 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:50.857061 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:50.858662 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:50.860382 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.860387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.860389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.860613 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.860844 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.860845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.860845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.860954 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.861245 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:50.861253 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:50.861408 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 21:02:50.861537 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 21:02:50.861716 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260810 21:02:50.862450 452 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:50.862541 452 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:50.862710 452 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:50.863283 32558 ts_manager.cc:194] Registered new tserver with Master: 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
I20260810 21:02:50.863894 32558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:58007
I20260810 21:02:50.864972 452 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.2@KRBTEST.COM
I20260810 21:02:50.871394 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.863595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40481 (local address 127.0.0.1:48422)
0810 21:02:50.863645 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:50.863654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:50.863689 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:50.864435 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:50.864437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:50.864719 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:50.864748 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.864750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.865798 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.865799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.866343 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.866345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.866798 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.866799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.866856 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.867355 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:50.867359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:50.868623 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:50.870278 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.870285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.870288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.870485 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.870703 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.870704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.870704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.870766 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.871064 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:50.871071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:50.871120 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 21:02:50.871207 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 21:02:50.871305 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 21:02:50.871408 458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.864081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.14.200.2:40481)
0810 21:02:50.864230 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.864234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.864265 (+ 31us) server_negotiation.cc:407] Connection header received
0810 21:02:50.864300 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.864302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.864318 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.864423 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.864872 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.865708 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.866423 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.866707 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.866793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.868720 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:50.868728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:50.868729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:50.868735 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:50.870194 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.870576 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.870579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.870579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.870639 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.870833 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.870834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.870834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.870968 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:50.871069 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.871245 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.871328 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260810 21:02:50.871773 379 tablet_service.cc:1514] Processing DeleteTablet for tablet bbc7717910f24a21a426bc53794f76ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:30 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:48422
I20260810 21:02:50.872200 459 tablet_replica.cc:330] stopping tablet replica
I20260810 21:02:50.872313 459 raft_consensus.cc:2229] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:50.872390 459 raft_consensus.cc:2258] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:50.872704 459 ts_tablet_manager.cc:1905] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:50.875082 459 ts_tablet_manager.cc:1918] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:50.875151 459 log.cc:1196] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/bbc7717910f24a21a426bc53794f76ae
I20260810 21:02:50.875387 459 ts_tablet_manager.cc:1939] T bbc7717910f24a21a426bc53794f76ae P 20ffc10c8ff144e888351475ba25e149: Deleting consensus metadata
I20260810 21:02:50.875865 32544 catalog_manager.cc:4841] TS 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): tablet bbc7717910f24a21a426bc53794f76ae (table test-table [id=d9302c86f7024185b605c673feda1abb]) successfully deleted
W20260810 21:02:50.944131 457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:02:50.944344 457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:02:50.944380 457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:02:50.945443 457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:02:50.945503 457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:02:50.945542 457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:02:50.946843 457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:36027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=34843
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:02:50.947114 457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:02:50.947341 457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:02:50.952178 465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.952188 466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:02:50.952188 468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:02:50.952381 457 server_base.cc:1034] running on GCE node
I20260810 21:02:50.952586 457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:02:50.952853 457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:02:50.954028 457 hybrid_clock.cc:648] HybridClock initialized: now 1786395770954018 us; error 51 us; skew 500 ppm
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:02:50.956419 457 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:02:50.956938 457 webserver.cc:460] Webserver started at http://127.14.200.3:34843/ using document root <none> and password file <none>
I20260810 21:02:50.957161 457 fs_manager.cc:359] Metadata directory not provided
I20260810 21:02:50.957212 457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:02:50.958446 457 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:02:50.958971 475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.959125 457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.959208 457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "fc0f62e39fa249e88c46177d2a24ce93"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "45bd3cc4a037b2c2c7305d50e3c72aea"
server_key_iv: "d3073072d220281913fe00b4f738ca00"
server_key_version: "encryptionkey@0"
I20260810 21:02:50.959483 457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:02:50.975203 457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:02:50.977100 457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:02:50.977206 457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:02:50.977391 457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:02:50.977650 457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:02:50.977692 457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.977720 457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:02:50.977741 457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:50.983492 457 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:36027
I20260810 21:02:50.983553 597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:36027 every 8 connection(s)
I20260810 21:02:50.983847 457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:02:50.987609 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 457
Aug 10 21:02:50 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395770, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:02:50.993505 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.984469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:54643 (local address 127.14.200.62:36683)
0810 21:02:50.984512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.984515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.984698 (+ 183us) server_negotiation.cc:407] Connection header received
0810 21:02:50.985272 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.985273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.985287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.985335 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.986191 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.986657 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.987732 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.987968 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.988047 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.990389 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:50.990392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:50.990392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:50.990400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:50.991906 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.992437 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.992439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.992439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.992504 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.992760 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.992761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.992761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.992954 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:50.993013 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.993296 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.993419 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:02:50.993646 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.984438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:54643)
0810 21:02:50.984578 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:50.984615 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:50.985169 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:50.985405 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:50.985409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:50.985929 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:02:50.986094 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.986098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.986770 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.986771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.987636 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:50.987638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.988187 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.988189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:50.988365 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.988854 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:02:50.988866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:02:50.990294 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:02:50.991998 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.992005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.992007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.992349 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.992580 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:02:50.992581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:02:50.992581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:02:50.992694 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:02:50.993019 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:02:50.993026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:02:50.993157 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 21:02:50.993259 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 21:02:50.993434 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260810 21:02:50.994187 598 heartbeater.cc:347] Connected to a master server at 127.14.200.62:36683
I20260810 21:02:50.994269 598 heartbeater.cc:464] Registering TS with master...
I20260810 21:02:50.994462 598 heartbeater.cc:511] Master 127.14.200.62:36683 requested a full tablet report, sending...
I20260810 21:02:50.994760 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.988062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.14.200.62:36683)
0810 21:02:50.988174 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.988176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.988254 (+ 78us) server_negotiation.cc:407] Connection header received
0810 21:02:50.988323 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.988323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.988349 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.988405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.989225 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.989692 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.990543 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.990729 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.990796 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.991889 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:50.991890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:50.991891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:50.991898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:50.993429 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.993843 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.993843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.993844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.993890 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:50.994127 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:50.994128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:50.994128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:50.994261 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:50.994435 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:50.994499 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 21:02:50.994587 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260810 21:02:50.994887 32558 ts_manager.cc:194] Registered new tserver with Master: fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027)
I20260810 21:02:50.995234 32558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:54643
I20260810 21:02:50.995827 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:02:50.998310 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.996177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.14.200.62:36683)
0810 21:02:50.996215 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.996217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.996330 (+ 113us) server_negotiation.cc:407] Connection header received
0810 21:02:50.996377 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.996378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.996403 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.996454 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:50.996528 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.997068 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.997755 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.997929 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:50.997976 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:50.998098 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:50.998132 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 21:02:50.998248 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39448: token expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 21:02:50.998365 602 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39448: token expired
W20260810 21:02:50.998543 32477 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.200.62:36683)
I20260810 21:02:51.003652 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:50.998769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.14.200.62:36683)
0810 21:02:50.998822 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:50.998824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:50.998830 (+ 6us) server_negotiation.cc:407] Connection header received
0810 21:02:50.998845 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:50.998846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:50.998852 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:50.998892 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:02:50.999280 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:50.999673 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.000267 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.000424 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.000468 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.001503 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:02:51.001504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:02:51.001504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:02:51.001510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:02:51.002586 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:51.002926 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:51.002927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:51.002928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:51.002984 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:02:51.003185 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:02:51.003186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:02:51.003186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:02:51.003287 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:02:51.003382 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:51.003485 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 21:02:51.003560 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:02:51.004869 32558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:02:51.006449 32558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:02:51.008592 32697 tablet_service.cc:1467] Processing CreateTablet for tablet c5efcd0720f448dd98efcc86248060bc (DEFAULT_TABLE table=test-table [id=e9de0d8fdee940b78b82e20f786ddc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:51.008733 379 tablet_service.cc:1467] Processing CreateTablet for tablet c5efcd0720f448dd98efcc86248060bc (DEFAULT_TABLE table=test-table [id=e9de0d8fdee940b78b82e20f786ddc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:51.008813 32697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5efcd0720f448dd98efcc86248060bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:51.008978 379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5efcd0720f448dd98efcc86248060bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:51.010295 32651 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Bootstrap starting.
I20260810 21:02:51.010520 333 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Bootstrap starting.
I20260810 21:02:51.010653 32651 tablet_bootstrap.cc:654] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:51.010831 333 tablet_bootstrap.cc:654] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:51.011237 32651 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: No bootstrap required, opened a new log
I20260810 21:02:51.011289 32651 ts_tablet_manager.cc:1397] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 21:02:51.011404 32651 raft_consensus.cc:348] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.011499 32651 raft_consensus.cc:374] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:51.011559 32651 raft_consensus.cc:729] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b3a10aaec457cb171273b2922da75, State: Initialized, Role: FOLLOWER
I20260810 21:02:51.011626 32651 consensus_queue.cc:260] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.011744 32651 ts_tablet_manager.cc:1428] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 21:02:51.011998 333 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: No bootstrap required, opened a new log
I20260810 21:02:51.012063 333 ts_tablet_manager.cc:1397] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:02:51.012224 333 raft_consensus.cc:348] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.012317 333 raft_consensus.cc:374] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:51.012384 333 raft_consensus.cc:729] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ffc10c8ff144e888351475ba25e149, State: Initialized, Role: FOLLOWER
I20260810 21:02:51.012450 333 consensus_queue.cc:260] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.012564 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.008276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:36027 (local address 127.0.0.1:34728)
0810 21:02:51.008325 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:51.008335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:51.008372 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:51.008862 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:51.008863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:51.008865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:51.008917 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.008918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.010210 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.010211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.011328 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.011332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.012331 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.012332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.012400 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.012404 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:02:51.012442 (+ 38us) client_negotiation.cc:240] Negotiation successful
0810 21:02:51.012486 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 21:02:51.012583 333 ts_tablet_manager.cc:1428] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 21:02:51.012923 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.008412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.14.200.3:36027)
0810 21:02:51.008596 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:51.008601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:51.008611 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:02:51.008645 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:51.008647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:51.008704 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:51.008942 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:51.009023 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.010062 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.011443 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.012217 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.012676 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.012749 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:51.012803 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 21:02:51.012855 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
I20260810 21:02:51.013635 528 tablet_service.cc:1467] Processing CreateTablet for tablet c5efcd0720f448dd98efcc86248060bc (DEFAULT_TABLE table=test-table [id=e9de0d8fdee940b78b82e20f786ddc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:02:51.013972 528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5efcd0720f448dd98efcc86248060bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:02:51.016522 607 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Bootstrap starting.
I20260810 21:02:51.017163 607 tablet_bootstrap.cc:654] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Neither blocks nor log segments found. Creating new log.
I20260810 21:02:51.017760 607 log.cc:824] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Log is configured to *not* fsync() on all Append() calls
I20260810 21:02:51.018437 607 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: No bootstrap required, opened a new log
I20260810 21:02:51.018517 607 ts_tablet_manager.cc:1397] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:51.019790 607 raft_consensus.cc:348] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.019925 607 raft_consensus.cc:374] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:02:51.019986 607 raft_consensus.cc:729] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0f62e39fa249e88c46177d2a24ce93, State: Initialized, Role: FOLLOWER
I20260810 21:02:51.020107 607 consensus_queue.cc:260] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.020373 607 ts_tablet_manager.cc:1428] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:02:51.020409 598 heartbeater.cc:503] Master 127.14.200.62:36683 was elected leader, sending a full tablet report...
W20260810 21:02:51.101898 453 tablet.cc:2369] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:51.201519 302 tablet.cc:2369] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:02:51.234328 599 tablet.cc:2369] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:02:51.335852 456 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:02:51.336030 456 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.336390 456 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:51.340318 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.336711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:57555 (local address 127.14.200.3:36027)
0810 21:02:51.336800 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:51.336803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:51.336816 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:02:51.336920 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:51.336922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:51.336933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:51.337021 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:51.337203 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.337965 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.339414 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.339933 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.340005 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.340022 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:51.340211 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 21:02:51.340249 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 21:02:51.340319 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.336954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:48409 (local address 127.14.200.1:46881)
0810 21:02:51.337084 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:51.337086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:51.337093 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:02:51.337120 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:51.337121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:51.337134 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:51.337220 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:02:51.337386 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.338086 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.339414 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.339933 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.340006 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.340013 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:51.340211 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 21:02:51.340249 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260810 21:02:51.340359 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.336616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:36027 (local address 127.14.200.2:57555)
0810 21:02:51.336717 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:51.336732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:51.336825 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:51.337022 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:51.337024 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:51.337026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:51.337119 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.337121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.338058 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.338059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.339320 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.339324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.340019 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.340021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.340098 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.340128 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 21:02:51.340193 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 21:02:51.340242 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":13}
I20260810 21:02:51.340705 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.336811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.14.200.2:48409)
0810 21:02:51.336965 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:51.336974 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:02:51.337018 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:02:51.337247 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:02:51.337248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:02:51.337250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:02:51.337307 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.337308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.338161 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.338161 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.339320 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:02:51.339324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.340037 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.340038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:02:51.340098 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.340128 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 21:02:51.340193 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 21:02:51.340242 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":35,"thread_start_us":105,"threads_started":1}
I20260810 21:02:51.340924 549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5efcd0720f448dd98efcc86248060bc" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0f62e39fa249e88c46177d2a24ce93" is_pre_election: true
I20260810 21:02:51.341101 549 raft_consensus.cc:2454] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 0.
I20260810 21:02:51.341053 32719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5efcd0720f448dd98efcc86248060bc" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b3a10aaec457cb171273b2922da75" is_pre_election: true
I20260810 21:02:51.341217 32719 raft_consensus.cc:2454] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 0.
I20260810 21:02:51.341377 327 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93; no voters:
I20260810 21:02:51.341514 456 raft_consensus.cc:2781] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:02:51.341602 456 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:02:51.341640 456 raft_consensus.cc:3037] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:51.342396 456 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.342538 456 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 election: Requested vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881)
I20260810 21:02:51.342679 549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5efcd0720f448dd98efcc86248060bc" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0f62e39fa249e88c46177d2a24ce93"
I20260810 21:02:51.342707 32719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5efcd0720f448dd98efcc86248060bc" candidate_uuid: "20ffc10c8ff144e888351475ba25e149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b3a10aaec457cb171273b2922da75"
I20260810 21:02:51.342772 549 raft_consensus.cc:3037] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:51.342782 32719 raft_consensus.cc:3037] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:02:51.343613 32719 raft_consensus.cc:2454] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 1.
I20260810 21:02:51.343699 549 raft_consensus.cc:2454] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20ffc10c8ff144e888351475ba25e149 in term 1.
I20260810 21:02:51.343797 327 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035b3a10aaec457cb171273b2922da75, 20ffc10c8ff144e888351475ba25e149; no voters:
I20260810 21:02:51.343901 456 raft_consensus.cc:2781] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:02:51.344070 456 raft_consensus.cc:686] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 1 LEADER]: Becoming Leader. State: Replica: 20ffc10c8ff144e888351475ba25e149, State: Running, Role: LEADER
I20260810 21:02:51.344162 456 consensus_queue.cc:237] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:02:51.345147 32558 catalog_manager.cc:5495] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20ffc10c8ff144e888351475ba25e149 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "20ffc10c8ff144e888351475ba25e149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } health_report { overall_health: HEALTHY } } }
I20260810 21:02:51.362859 458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.360671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.14.200.2:40481)
0810 21:02:51.360736 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:02:51.360737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:02:51.360743 (+ 6us) server_negotiation.cc:407] Connection header received
0810 21:02:51.360769 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:02:51.360803 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:02:51.360821 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:02:51.360883 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:02:51.360992 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.361472 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.362139 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:02:51.362315 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:02:51.362362 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:02:51.362553 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:02:51.362647 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:02:51.362729 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 21:02:51.362769 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 21:02:51.362790 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 21:02:51.364167 549 raft_consensus.cc:1261] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Refusing update from remote peer 20ffc10c8ff144e888351475ba25e149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:51.364240 32719 raft_consensus.cc:1261] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Refusing update from remote peer 20ffc10c8ff144e888351475ba25e149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:02:51.364557 456 consensus_queue.cc:1035] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:51.364686 613 consensus_queue.cc:1035] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:02:51.367144 616 mvcc.cc:205] Tried to move back new op lower bound from 7317077079505612800 to 7317077079426490368. Current Snapshot: MvccSnapshot[applied={T|T < 7317077079505612800}]
I20260810 21:02:51.367417 617 mvcc.cc:205] Tried to move back new op lower bound from 7317077079505612800 to 7317077079426490368. Current Snapshot: MvccSnapshot[applied={T|T < 7317077079505612800}]
I20260810 21:02:51.369426 32557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:51.369521 32557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:02:51.370255 32557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Sending DeleteTablet for 3 replicas of tablet c5efcd0720f448dd98efcc86248060bc
I20260810 21:02:51.370560 32697 tablet_service.cc:1514] Processing DeleteTablet for tablet c5efcd0720f448dd98efcc86248060bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:51 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:46632
I20260810 21:02:51.370608 379 tablet_service.cc:1514] Processing DeleteTablet for tablet c5efcd0720f448dd98efcc86248060bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:51 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:48422
I20260810 21:02:51.370602 528 tablet_service.cc:1514] Processing DeleteTablet for tablet c5efcd0720f448dd98efcc86248060bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:02:51 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:34728
I20260810 21:02:51.370726 459 tablet_replica.cc:330] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: stopping tablet replica
I20260810 21:02:51.370746 627 tablet_replica.cc:330] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: stopping tablet replica
I20260810 21:02:51.370795 459 raft_consensus.cc:2229] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:02:51.370801 627 raft_consensus.cc:2229] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:51.370843 627 raft_consensus.cc:2258] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:51.370884 459 raft_consensus.cc:2258] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:51.370913 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 32601
I20260810 21:02:51.371008 628 tablet_replica.cc:330] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: stopping tablet replica
I20260810 21:02:51.371140 628 raft_consensus.cc:2229] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:02:51.371238 459 ts_tablet_manager.cc:1905] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:51.371348 628 raft_consensus.cc:2258] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:02:51.372649 628 ts_tablet_manager.cc:1905] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:02:51.372845 459 ts_tablet_manager.cc:1918] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:51.372906 459 log.cc:1196] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/c5efcd0720f448dd98efcc86248060bc
I20260810 21:02:51.373148 459 ts_tablet_manager.cc:1939] T c5efcd0720f448dd98efcc86248060bc P 20ffc10c8ff144e888351475ba25e149: Deleting consensus metadata
I20260810 21:02:51.373500 32544 catalog_manager.cc:4841] TS 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): tablet c5efcd0720f448dd98efcc86248060bc (table test-table [id=e9de0d8fdee940b78b82e20f786ddc9e]) successfully deleted
I20260810 21:02:51.374305 628 ts_tablet_manager.cc:1918] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:02:51.374382 628 log.cc:1196] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/c5efcd0720f448dd98efcc86248060bc
I20260810 21:02:51.374684 628 ts_tablet_manager.cc:1939] T c5efcd0720f448dd98efcc86248060bc P fc0f62e39fa249e88c46177d2a24ce93: Deleting consensus metadata
I20260810 21:02:51.375078 32541 catalog_manager.cc:4841] TS fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): tablet c5efcd0720f448dd98efcc86248060bc (table test-table [id=e9de0d8fdee940b78b82e20f786ddc9e]) successfully deleted
W20260810 21:02:51.382961 32541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:46881 (error 108)
I20260810 21:02:51.383395 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 306
I20260810 21:02:51.383430 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:02:51.383313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46881 (local address 127.0.0.1:46644)
0810 21:02:51.383360 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 21:02:51.383397 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 21:02:51.383529 32541 catalog_manager.cc:4584] TS 035b3a10aaec457cb171273b2922da75 (127.14.200.1:46881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5efcd0720f448dd98efcc86248060bc: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46881: connect: Connection refused (error 111)
I20260810 21:02:51.393410 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 457
I20260810 21:02:51.399823 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 32521
I20260810 21:03:11.406899 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--webserver_interface=127.14.200.62
--webserver_port=39211
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:11.487274 630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:11.487463 630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:11.487483 630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:11.488641 630 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 21:03:11.488682 630 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 21:03:11.488695 630 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 21:03:11.488708 630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:11.488720 630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:11.488796 630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:11.488816 630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:11.488837 630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:11.490310 630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=39211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:11.490567 630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:11.490799 630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:11.495689 635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.495767 630 server_base.cc:1034] running on GCE node
W20260810 21:03:11.495980 638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:11.495986 636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.496317 630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:11.496659 630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:11.497839 630 hybrid_clock.cc:648] HybridClock initialized: now 1786395791497825 us; error 47 us; skew 500 ppm
Aug 10 21:03:11 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:11.500375 630 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:11.500905 630 webserver.cc:460] Webserver started at http://127.14.200.62:39211/ using document root <none> and password file <none>
I20260810 21:03:11.501139 630 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:11.501185 630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:11.502445 630 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:11.503115 645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.503314 630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:11.503408 630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f8710528382c47aba5afcf09d7b5bf5e"
format_stamp: "Formatted at 2026-08-10 21:02:28 on dist-test-slave-1x4g"
server_key: "15ebc9b35ced6b77f4f2e78c272558cd"
server_key_iv: "2569b6281be0fe757286405099d66f35"
server_key_version: "encryptionkey@0"
I20260810 21:03:11.503719 630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:11.514986 630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:11.517279 630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:11.517453 630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:11.521634 630 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36683
I20260810 21:03:11.521667 700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36683 every 8 connection(s)
I20260810 21:03:11.522037 630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:11.523098 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 630
I20260810 21:03:11.523404 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:46881
--local_ip_for_outbound_sockets=127.14.200.1
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=45513
--webserver_interface=127.14.200.1
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:03:11.523564 701 sys_catalog.cc:263] Verifying existing consensus state
I20260810 21:03:11.524219 701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap starting.
I20260810 21:03:11.533663 701 log.cc:824] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:11.535868 701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:03:11.536185 701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Bootstrap complete.
I20260810 21:03:11.538650 701 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:03:11.538900 701 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Initialized, Role: FOLLOWER
I20260810 21:03:11.539119 701 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:03:11.539225 701 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:11.539280 701 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:11.539328 701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 2 FOLLOWER]: Advancing to term 3
I20260810 21:03:11.540267 701 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:03:11.540392 701 leader_election.cc:302] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8710528382c47aba5afcf09d7b5bf5e; no voters:
I20260810 21:03:11.540570 701 leader_election.cc:288] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 21:03:11.540642 706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 3 FOLLOWER]: Leader election won for term 3
I20260810 21:03:11.540798 706 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [term 3 LEADER]: Becoming Leader. State: Replica: f8710528382c47aba5afcf09d7b5bf5e, State: Running, Role: LEADER
I20260810 21:03:11.540851 701 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:11.540982 706 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } }
I20260810 21:03:11.541282 708 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8710528382c47aba5afcf09d7b5bf5e. Latest consensus state: current_term: 3 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:03:11.541345 708 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:11.541661 711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:11.541673 707 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f8710528382c47aba5afcf09d7b5bf5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8710528382c47aba5afcf09d7b5bf5e" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36683 } } }
I20260810 21:03:11.541836 707 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:11.542618 711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:11.542781 711 catalog_manager.cc:1259] Loaded cluster ID: c60b2e52db82433c93745a6fd47830e6
I20260810 21:03:11.542997 711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:11.543743 711 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:11.543848 711 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Loaded TSK: 0
I20260810 21:03:11.543869 711 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f8710528382c47aba5afcf09d7b5bf5e: Loaded TSK: 1
F20260810 21:03:11.543999 711 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb66045a9ad google::LogMessage::Fail() at ??:0
@ 0x7fb66045d88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb66045a4de google::LogMessage::Flush() at ??:0
@ 0x7fb66045ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb662bc7104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb662bc4d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb6683b41cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb6683b448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb6683b4594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb66837ac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb668384ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb6623280d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb66231d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb6631816db start_thread at ??:0
@ 0x7fb65ef5861f clone at ??:0
W20260810 21:03:11.621858 704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:11.622090 704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:11.622133 704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:11.623261 704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:11.623330 704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:11.623374 704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:11.624864 704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:46881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=45513
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:11.625128 704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:11.625348 704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:11.631152 746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.631230 704 server_base.cc:1034] running on GCE node
W20260810 21:03:11.631165 747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:11.631371 749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.631608 704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:11.631870 704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:11.633044 704 hybrid_clock.cc:648] HybridClock initialized: now 1786395791633032 us; error 28 us; skew 500 ppm
Aug 10 21:03:11 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:11.635468 704 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:03:11.635975 704 webserver.cc:460] Webserver started at http://127.14.200.1:45513/ using document root <none> and password file <none>
I20260810 21:03:11.636219 704 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:11.636264 704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:11.637692 704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:11.638409 756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.638594 704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:11.638720 704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "035b3a10aaec457cb171273b2922da75"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "f0c420fc6243c6f52eec73c254a29a42"
server_key_iv: "df8636d19a4f543b22c3bf7dde93ef64"
server_key_version: "encryptionkey@0"
I20260810 21:03:11.639108 704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:11.650758 704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:11.653004 704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:11.653144 704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:11.653393 704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:11.653852 764 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 21:03:11.655304 704 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 21:03:11.655359 704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 21:03:11.655383 704 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 21:03:11.656030 704 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 21:03:11.656075 704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 21:03:11.656127 764 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Bootstrap starting.
I20260810 21:03:11.663865 704 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46881
I20260810 21:03:11.664356 704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:11.667817 764 log.cc:824] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:11.668102 883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46881 every 8 connection(s)
I20260810 21:03:11.669533 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 704
I20260810 21:03:11.669881 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:40481
--local_ip_for_outbound_sockets=127.14.200.2
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=44541
--webserver_interface=127.14.200.2
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:03:11.670168 764 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 21:03:11.670433 764 tablet_bootstrap.cc:492] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Bootstrap complete.
I20260810 21:03:11.671409 764 ts_tablet_manager.cc:1397] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.001s
I20260810 21:03:11.673086 764 raft_consensus.cc:348] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:11.673395 764 raft_consensus.cc:729] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b3a10aaec457cb171273b2922da75, State: Initialized, Role: FOLLOWER
I20260810 21:03:11.673542 764 consensus_queue.cc:260] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:11.673810 764 ts_tablet_manager.cc:1428] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75: Time spent starting tablet: real 0.002s user 0.001s sys 0.004s
W20260810 21:03:11.759897 887 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:03:11.665119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:36609)
0810 21:03:11.665341 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.665404 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:11.666122 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:11.759732 (+ 93610us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
W20260810 21:03:11.760221 885 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36683 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36683: Client connection negotiation failed: client connection to 127.14.200.62:36683: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:03:11.760676 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.760518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:41433)
0810 21:03:11.760585 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.760645 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 21:03:11.761060 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.760952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:42367)
0810 21:03:11.761009 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.761039 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 21:03:11.761198 885 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 21:03:11.773803 889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:11.773972 889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:11.773993 889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:11.775067 889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:11.775107 889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:11.775163 889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:11.776738 889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:40481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=44541
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:11.777024 889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:11.777298 889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:11.782102 895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:11.782128 898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:11.782128 896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.782266 889 server_base.cc:1034] running on GCE node
I20260810 21:03:11.782444 889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:11.782629 889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:11.783764 889 hybrid_clock.cc:648] HybridClock initialized: now 1786395791783752 us; error 27 us; skew 500 ppm
Aug 10 21:03:11 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:11.786079 889 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:03:11.786439 889 webserver.cc:460] Webserver started at http://127.14.200.2:44541/ using document root <none> and password file <none>
I20260810 21:03:11.786624 889 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:11.786701 889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:11.787946 889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:03:11.788506 906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.788702 889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:11.788818 889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "20ffc10c8ff144e888351475ba25e149"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "c554f59399618c77e8c1a4005ce47f70"
server_key_iv: "38f1a33e69ce301db735e8e0f645ddb7"
server_key_version: "encryptionkey@0"
I20260810 21:03:11.789155 889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:11.808229 889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:11.810629 889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:11.810792 889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:11.811033 889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:11.811331 889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:11.811386 889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.811424 889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:11.811444 889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.817195 889 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:40481
I20260810 21:03:11.817250 1033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:40481 every 8 connection(s)
I20260810 21:03:11.817621 889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:03:11.818818 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.818425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:44313)
0810 21:03:11.818595 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.818681 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
W20260810 21:03:11.819248 1034 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36683 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36683: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
I20260810 21:03:11.819521 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.819435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:44081)
0810 21:03:11.819479 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.819505 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 21:03:11.819756 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.819708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:43685)
0810 21:03:11.819728 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.819745 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260810 21:03:11.819875 1034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 21:03:11.827323 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 889
I20260810 21:03:11.827622 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:36027
--local_ip_for_outbound_sockets=127.14.200.3
--tserver_master_addrs=127.14.200.62:36683
--webserver_port=34843
--webserver_interface=127.14.200.3
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:11.909669 1037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:11.909865 1037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:11.909885 1037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:11.910905 1037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:11.910943 1037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:11.910965 1037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:11.912194 1037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:36027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=34843
--tserver_master_addrs=127.14.200.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:11.912356 1037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:11.912508 1037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:11.917379 1044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.917385 1037 server_base.cc:1034] running on GCE node
W20260810 21:03:11.917337 1046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:11.917352 1043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:11.917729 1037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:11.917991 1037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:11.919157 1037 hybrid_clock.cc:648] HybridClock initialized: now 1786395791919143 us; error 33 us; skew 500 ppm
Aug 10 21:03:11 dist-test-slave-1x4g krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:11.921595 1037 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:03:11.921993 1037 webserver.cc:460] Webserver started at http://127.14.200.3:34843/ using document root <none> and password file <none>
I20260810 21:03:11.922201 1037 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:11.922243 1037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:11.923622 1037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:11.924196 1054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.924371 1037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:11.924484 1037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "fc0f62e39fa249e88c46177d2a24ce93"
format_stamp: "Formatted at 2026-08-10 21:02:29 on dist-test-slave-1x4g"
server_key: "45bd3cc4a037b2c2c7305d50e3c72aea"
server_key_iv: "d3073072d220281913fe00b4f738ca00"
server_key_version: "encryptionkey@0"
I20260810 21:03:11.924786 1037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:11.934556 1037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:11.936364 1037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:11.936529 1037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:11.936803 1037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:11.937127 1037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:11.937186 1037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.937219 1037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:11.937247 1037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:11.943233 1037 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:36027
I20260810 21:03:11.943336 1176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:36027 every 8 connection(s)
I20260810 21:03:11.943647 1037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:03:11.944695 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.944329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:47657)
0810 21:03:11.944492 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.944569 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
W20260810 21:03:11.945086 1177 heartbeater.cc:644] Failed to heartbeat to 127.14.200.62:36683 (0 consecutive failures): Network error: Failed to ping master at 127.14.200.62:36683: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
I20260810 21:03:11.945384 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.945286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:42165)
0810 21:03:11.945342 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.945367 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 21:03:11.945662 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:11.945613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:54187)
0810 21:03:11.945632 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:11.945650 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260810 21:03:11.945808 1177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 21:03:11.953270 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1037
I20260810 21:03:12.761842 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:12.761589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:51531)
0810 21:03:12.761735 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:12.761772 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260810 21:03:12.820487 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:12.820201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:55683)
0810 21:03:12.820386 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:12.820417 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260810 21:03:12.946298 1183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:12.946106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:41397)
0810 21:03:12.946216 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:12.946244 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260810 21:03:13.501106 1184 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:13.501271 1184 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:13.501641 1184 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
Aug 10 21:03:13 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.3@KRBTEST.COM
Aug 10 21:03:13 dist-test-slave-1x4g krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.2@KRBTEST.COM
I20260810 21:03:13.511981 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.501788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:36027 (local address 127.14.200.1:35735)
0810 21:03:13.501912 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:13.501924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:13.501986 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:13.502819 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:13.502822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:13.503179 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:13.503356 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:13.503362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.504446 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.504447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:13.505257 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:13.505260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.505835 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:13.505954 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:13.506540 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:13.506548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:13.507889 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:13.510200 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:13.510208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:13.510499 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:13.510779 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:13.510779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:13.510928 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:13.511408 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:13.511412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:13.511576 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 21:03:13.511687 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 21:03:13.511878 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":62,"thread_start_us":55,"threads_started":1}
I20260810 21:03:13.511991 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.501934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:37725 (local address 127.14.200.2:40481)
0810 21:03:13.502115 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:13.502125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:13.502145 (+ 20us) server_negotiation.cc:407] Connection header received
0810 21:03:13.502572 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:13.502574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:13.502614 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:13.502766 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:13.503509 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.504328 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505353 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.505734 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505867 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:13.508370 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:13.508375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:13.508375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:13.508386 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:13.510336 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510697 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:13.510700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:13.510700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:13.510800 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:13.511021 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:13.511022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:13.511022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:13.511319 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:13.511407 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:13.511764 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 21:03:13.511888 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":58,"threads_started":1}
I20260810 21:03:13.511991 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.501905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:35735 (local address 127.14.200.3:36027)
0810 21:03:13.502132 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:13.502140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:13.502159 (+ 19us) server_negotiation.cc:407] Connection header received
0810 21:03:13.502572 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:13.502574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:13.502614 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:13.502768 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:13.503482 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.504329 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505353 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.505736 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505860 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:13.508007 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:13.508010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:13.508011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:13.508027 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:13.510120 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510585 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:13.510587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:13.510588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:13.510705 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:13.511028 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:13.511028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:13.511029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:13.511319 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:13.511407 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:13.511764 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 21:03:13.511888 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":58,"threads_started":1}
I20260810 21:03:13.512055 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.501805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40481 (local address 127.14.200.1:37725)
0810 21:03:13.501947 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:13.501958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:13.502014 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:13.502818 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:13.502821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:13.503179 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:13.503356 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:13.503362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.504440 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.504442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:13.505259 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:13.505262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:13.505834 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:13.505835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:13.505959 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:13.506603 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:13.506608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:13.508269 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:13.510412 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:13.510413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:13.510624 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:13.510873 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:13.510873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:13.510874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:13.510952 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:13.511403 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:13.511411 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:13.511576 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 21:03:13.511687 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 21:03:13.511948 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":147,"thread_start_us":72,"threads_started":1}
W20260810 21:03:13.513048 757 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
W20260810 21:03:13.513162 760 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
I20260810 21:03:13.513199 757 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 035b3a10aaec457cb171273b2922da75; no voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93
I20260810 21:03:13.513306 1184 raft_consensus.cc:2726] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:03:13.762485 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.762348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:59745)
0810 21:03:13.762408 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:13.762443 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 21:03:13.821193 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.820974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:42691)
0810 21:03:13.821104 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:13.821151 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":55,"threads_started":1}
I20260810 21:03:13.946959 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:13.946727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:43301)
0810 21:03:13.946860 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:13.946912 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260810 21:03:14.763235 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:14.763030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:54699)
0810 21:03:14.763157 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:14.763190 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260810 21:03:14.821817 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:14.821625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:35171)
0810 21:03:14.821750 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:14.821781 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260810 21:03:14.947566 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:14.947385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:41241)
0810 21:03:14.947500 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:14.947529 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260810 21:03:15.762387 1194 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:15.762485 1194 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:15.762677 1194 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
W20260810 21:03:15.763118 757 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
W20260810 21:03:15.763203 760 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
I20260810 21:03:15.763274 760 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 035b3a10aaec457cb171273b2922da75; no voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93
I20260810 21:03:15.763387 1194 raft_consensus.cc:2726] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:03:15.763833 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:15.763621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:37929)
0810 21:03:15.763740 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:15.763770 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260810 21:03:15.822453 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:15.822224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:44955)
0810 21:03:15.822358 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:15.822390 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260810 21:03:15.948185 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:15.947960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:38389)
0810 21:03:15.948103 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:15.948136 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260810 21:03:16.764665 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:16.764362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:54077)
0810 21:03:16.764577 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:16.764617 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":62,"threads_started":1}
I20260810 21:03:16.823175 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:16.822972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:47435)
0810 21:03:16.823100 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:16.823135 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260810 21:03:16.948977 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:16.948735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:59249)
0810 21:03:16.948907 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:16.948937 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260810 21:03:17.491731 1201 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:17.491858 1201 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:17.492100 1201 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
W20260810 21:03:17.492492 760 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
W20260810 21:03:17.492563 757 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
I20260810 21:03:17.492621 757 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 035b3a10aaec457cb171273b2922da75; no voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93
I20260810 21:03:17.492708 1201 raft_consensus.cc:2726] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:03:17.765334 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:17.765112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:43379)
0810 21:03:17.765251 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:17.765286 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260810 21:03:17.823899 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:17.823694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:43875)
0810 21:03:17.823824 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:17.823863 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260810 21:03:17.949604 1204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:17.949411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:34347)
0810 21:03:17.949534 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:17.949565 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260810 21:03:18.766047 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:18.765826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:34949)
0810 21:03:18.765962 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:18.766000 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260810 21:03:18.824519 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:18.824329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:52709)
0810 21:03:18.824449 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:18.824480 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260810 21:03:18.950336 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:18.950106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:56533)
0810 21:03:18.950246 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:18.950287 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260810 21:03:19.766860 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:19.766607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:54557)
0810 21:03:19.766758 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:19.766809 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260810 21:03:19.825276 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:19.825031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:41003)
0810 21:03:19.825166 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:19.825236 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260810 21:03:19.950970 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:19.950786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:46999)
0810 21:03:19.950903 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:19.950932 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260810 21:03:20.767590 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:20.767388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:48591)
0810 21:03:20.767511 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:20.767544 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260810 21:03:20.825842 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:20.825659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:46879)
0810 21:03:20.825776 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:20.825806 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260810 21:03:20.951577 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:20.951379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:42165)
0810 21:03:20.951509 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:20.951540 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260810 21:03:21.369338 1217 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:21.369442 1217 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:21.369608 1217 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
W20260810 21:03:21.370055 760 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
W20260810 21:03:21.370138 757 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
I20260810 21:03:21.370174 757 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 035b3a10aaec457cb171273b2922da75; no voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93
I20260810 21:03:21.370261 1217 raft_consensus.cc:2726] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:03:21.768380 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:21.768087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:51557)
0810 21:03:21.768294 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:21.768331 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":62,"threads_started":1}
I20260810 21:03:21.826463 1219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:21.826250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:38605)
0810 21:03:21.826368 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:21.826400 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260810 21:03:21.952334 1220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:21.952114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:44987)
0810 21:03:21.952235 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:21.952289 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260810 21:03:22.769198 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:22.768987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:36429)
0810 21:03:22.769117 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:22.769151 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260810 21:03:22.827384 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:22.827086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:36363)
0810 21:03:22.827258 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:22.827322 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":59,"threads_started":1}
I20260810 21:03:22.953043 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:22.952861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:46857)
0810 21:03:22.952971 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:22.953002 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260810 21:03:23.251695 1224 raft_consensus.cc:482] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:23.251806 1224 raft_consensus.cc:504] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0f62e39fa249e88c46177d2a24ce93" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 36027 } } peers { permanent_uuid: "035b3a10aaec457cb171273b2922da75" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46881 } } peers { permanent_uuid: "20ffc10c8ff144e888351475ba25e149" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40481 } }
I20260810 21:03:23.252032 1224 leader_election.cc:288] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027), 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481)
W20260810 21:03:23.252354 757 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fc0f62e39fa249e88c46177d2a24ce93 (127.14.200.3:36027): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
W20260810 21:03:23.252445 760 leader_election.cc:341] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20ffc10c8ff144e888351475ba25e149 (127.14.200.2:40481): Not found: Tablet not found: c5efcd0720f448dd98efcc86248060bc
I20260810 21:03:23.252525 760 leader_election.cc:302] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 035b3a10aaec457cb171273b2922da75; no voters: 20ffc10c8ff144e888351475ba25e149, fc0f62e39fa249e88c46177d2a24ce93
I20260810 21:03:23.252605 1224 raft_consensus.cc:2726] T c5efcd0720f448dd98efcc86248060bc P 035b3a10aaec457cb171273b2922da75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 21:03:23.769865 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:23.769661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:33959)
0810 21:03:23.769784 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:23.769818 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260810 21:03:23.827967 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:23.827786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:60409)
0810 21:03:23.827891 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:23.827923 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260810 21:03:23.953639 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:23.953444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:46151)
0810 21:03:23.953568 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:23.953602 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":55,"threads_started":1}
I20260810 21:03:24.770589 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:24.770342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:47845)
0810 21:03:24.770486 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:24.770526 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260810 21:03:24.828585 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:24.828341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:41959)
0810 21:03:24.828488 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:24.828526 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260810 21:03:24.954268 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:24.954069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:44581)
0810 21:03:24.954191 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:24.954226 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260810 21:03:25.771370 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:25.771148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:51817)
0810 21:03:25.771290 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:25.771322 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260810 21:03:25.829252 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:25.829004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:43167)
0810 21:03:25.829157 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:25.829208 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260810 21:03:25.954857 1233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:25.954677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:41331)
0810 21:03:25.954786 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:25.954818 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260810 21:03:26.772181 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:26.771942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.1:50953)
0810 21:03:26.772099 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:26.772134 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260810 21:03:26.830118 1235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:26.829878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.2:53081)
0810 21:03:26.830015 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:26.830066 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260810 21:03:26.955628 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:26.955347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:36683 (local address 127.14.200.3:34355)
0810 21:03:26.955560 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:26.955592 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.62:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":72,"threads_started":1}
I20260810 21:03:26.958595 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 704
I20260810 21:03:26.965662 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 889
I20260810 21:03:26.972281 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1037
2026-08-10T21:03:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59620 ms)
[----------] 30 tests from SecurityITest (232663 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:27 dist-test-slave-1x4g krb5kdc[1239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:27 dist-test-slave-1x4g krb5kdc[1239](info): set up 2 sockets
Aug 10 21:03:27 dist-test-slave-1x4g krb5kdc[1239](info): commencing operation
krb5kdc: starting...
W20260810 21:03:28.030821 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:28.149421 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45917
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:28.228402 1253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:28.228585 1253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:28.228605 1253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:28.229787 1253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:28.229830 1253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:28.229844 1253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:28.229857 1253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:28.229869 1253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:28.231348 1253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:28.231525 1253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:28.231684 1253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:28.236716 1253 server_base.cc:1034] running on GCE node
W20260810 21:03:28.236805 1259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.236896 1258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.236904 1261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.237208 1253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:28.237527 1253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:28.238690 1253 hybrid_clock.cc:648] HybridClock initialized: now 1786395808238675 us; error 43 us; skew 500 ppm
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:28.241051 1253 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:28.241487 1253 webserver.cc:460] Webserver started at http://127.14.200.62:38343/ using document root <none> and password file <none>
I20260810 21:03:28.241719 1253 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:28.241786 1253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:28.241930 1253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:28.242936 1253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "f1cd76b366e74a5c879c4166afe4948b"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "cc73952b853bbaa0ec7457202213977c"
server_key_iv: "eb5630ade595556c0e3cb208fb93f00b"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.243286 1253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "f1cd76b366e74a5c879c4166afe4948b"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "cc73952b853bbaa0ec7457202213977c"
server_key_iv: "eb5630ade595556c0e3cb208fb93f00b"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.244614 1253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:28.245384 1268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.245553 1253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.245651 1253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f1cd76b366e74a5c879c4166afe4948b"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "cc73952b853bbaa0ec7457202213977c"
server_key_iv: "eb5630ade595556c0e3cb208fb93f00b"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.245731 1253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:28.254503 1253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:28.256124 1253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:28.256274 1253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:28.260342 1253 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45917
I20260810 21:03:28.260375 1325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45917 every 8 connection(s)
I20260810 21:03:28.260699 1253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:28.261647 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:28.263856 1326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: Bootstrap starting.
I20260810 21:03:28.264287 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1253
I20260810 21:03:28.264406 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:28.264535 1326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:28.264631 15136 external_mini_cluster.cc:1428] Setting key e659bf01af11908ac65e7d0a0839bd56
I20260810 21:03:28.264959 1326 log.cc:824] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:28.265744 1326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: No bootstrap required, opened a new log
I20260810 21:03:28.267611 1326 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } }
I20260810 21:03:28.267777 1326 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:28.267827 1326 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cd76b366e74a5c879c4166afe4948b, State: Initialized, Role: FOLLOWER
I20260810 21:03:28.268025 1326 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } }
I20260810 21:03:28.268146 1332 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:28.268247 1332 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } }
I20260810 21:03:28.268127 1326 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:28.268369 1332 leader_election.cc:302] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1cd76b366e74a5c879c4166afe4948b; no voters:
I20260810 21:03:28.268406 1326 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:28.268437 1326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:28.268541 1332 leader_election.cc:288] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260810 21:03:28.269147 1326 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } }
I20260810 21:03:28.269250 1326 leader_election.cc:302] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1cd76b366e74a5c879c4166afe4948b; no voters:
I20260810 21:03:28.269265 1333 raft_consensus.cc:2741] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 21:03:28.269362 1326 leader_election.cc:288] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:28.269443 1332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:28.269543 1332 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [term 1 LEADER]: Becoming Leader. State: Replica: f1cd76b366e74a5c879c4166afe4948b, State: Running, Role: LEADER
I20260810 21:03:28.269627 1332 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } }
I20260810 21:03:28.269636 1326 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:28.270134 1334 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1cd76b366e74a5c879c4166afe4948b. Latest consensus state: current_term: 1 leader_uuid: "f1cd76b366e74a5c879c4166afe4948b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } } }
I20260810 21:03:28.270226 1334 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:28.270475 1333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1cd76b366e74a5c879c4166afe4948b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cd76b366e74a5c879c4166afe4948b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45917 } } }
I20260810 21:03:28.270613 1333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:28.270828 1337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:28.271404 1337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:28.273161 1337 catalog_manager.cc:1347] Generated new cluster ID: e00772a6a3df44a695db098a29ba964e
I20260810 21:03:28.273231 1337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:28.276048 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.265209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.14.200.62:45917)
0810 21:03:28.265352 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.265360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.265393 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:03:28.265909 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.265915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.265973 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.266116 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:28.266725 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.267577 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.268550 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.269013 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.269139 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.272047 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:28.272052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:28.272052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:28.272065 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:28.274350 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.274775 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.274778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.274778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.274906 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.275120 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.275121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.275121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.275446 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:28.275619 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.275697 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.275837 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260810 21:03:28.290884 1337 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:28.291608 1337 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:28.297155 1337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: Generated new TSK 0
I20260810 21:03:28.297341 1337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:03:28.323516 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:28.403870 1355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:28.404119 1355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:28.404155 1355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:28.405273 1355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:28.405339 1355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:28.405383 1355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:28.406785 1355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:28.407042 1355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:28.407263 1355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:28.412213 1361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.412222 1362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.412230 1364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.412446 1355 server_base.cc:1034] running on GCE node
I20260810 21:03:28.412698 1355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:28.412949 1355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:28.414089 1355 hybrid_clock.cc:648] HybridClock initialized: now 1786395808414076 us; error 27 us; skew 500 ppm
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:28.416561 1355 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:03:28.417064 1355 webserver.cc:460] Webserver started at http://127.14.200.1:36581/ using document root <none> and password file <none>
I20260810 21:03:28.417248 1355 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:28.417287 1355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:28.417374 1355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:28.418318 1355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "231cbccacdab4a61891413d8e78b50f4"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "15ed0bbdd4b13a03aca78866786636ad"
server_key_iv: "a26d974076c1b7d2e729f3942e589ce7"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.418649 1355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "231cbccacdab4a61891413d8e78b50f4"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "15ed0bbdd4b13a03aca78866786636ad"
server_key_iv: "a26d974076c1b7d2e729f3942e589ce7"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.419978 1355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:28.420706 1373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.420886 1355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:28.420990 1355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "231cbccacdab4a61891413d8e78b50f4"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "15ed0bbdd4b13a03aca78866786636ad"
server_key_iv: "a26d974076c1b7d2e729f3942e589ce7"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.421082 1355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:28.434162 1355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:28.435798 1355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:28.435950 1355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:28.436170 1355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:28.436467 1355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:28.436520 1355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.436559 1355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:28.436586 1355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.442842 1355 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:41237
I20260810 21:03:28.442891 1498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:41237 every 8 connection(s)
I20260810 21:03:28.443156 1355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:28.448580 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1355
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:28.448712 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:28.448961 15136 external_mini_cluster.cc:1428] Setting key 3fc72197fe9b1029868da24c524c1c87
I20260810 21:03:28.453954 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.443874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53171 (local address 127.14.200.62:45917)
0810 21:03:28.443940 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.443942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.444108 (+ 166us) server_negotiation.cc:407] Connection header received
0810 21:03:28.444548 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.444549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.444560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.444637 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:28.445524 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.446011 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.446899 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.447074 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.447127 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.449487 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:28.449489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:28.449489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:28.449496 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:28.452468 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.452975 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.452976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.452977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.453030 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.453338 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.453339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.453339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.453454 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:28.453504 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.453816 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.453886 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:03:28.454244 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.443830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45917 (local address 127.14.200.1:53171)
0810 21:03:28.443985 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.444023 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.444465 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.444684 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.444687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.445265 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:28.445440 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.445444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.446094 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.446094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.446828 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.446830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.447146 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.447147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.447254 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.447673 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:28.447681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:28.449387 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:28.452572 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.452579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.452582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.452893 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.453096 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.453097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.453097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.453247 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.453515 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:28.453521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:28.453675 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.453783 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.453965 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260810 21:03:28.454810 1499 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45917
I20260810 21:03:28.454914 1499 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:28.455096 1499 heartbeater.cc:511] Master 127.14.200.62:45917 requested a full tablet report, sending...
I20260810 21:03:28.455574 1286 ts_manager.cc:194] Registered new tserver with Master: 231cbccacdab4a61891413d8e78b50f4 (127.14.200.1:41237)
I20260810 21:03:28.456148 1286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:53171
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:03:28.475198 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:28.555207 1504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:28.555399 1504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:28.555424 1504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:28.556529 1504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:28.556567 1504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:28.556612 1504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:28.557972 1504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:28.558161 1504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:28.558331 1504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:28.563141 1511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.563292 1504 server_base.cc:1034] running on GCE node
W20260810 21:03:28.563287 1510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.563516 1513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.563767 1504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:28.563988 1504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:28.565168 1504 hybrid_clock.cc:648] HybridClock initialized: now 1786395808565153 us; error 28 us; skew 500 ppm
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:28.567682 1504 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:03:28.568182 1504 webserver.cc:460] Webserver started at http://127.14.200.2:46601/ using document root <none> and password file <none>
I20260810 21:03:28.568403 1504 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:28.568450 1504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:28.568621 1504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:28.569759 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "8d58f87f9f644feb827add4fd5d98197"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "3ad7b44ffad03f7428e5ff382ce70b91"
server_key_iv: "02c7d23c9323c739de398d7f3b1b60d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.570170 1504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "8d58f87f9f644feb827add4fd5d98197"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "3ad7b44ffad03f7428e5ff382ce70b91"
server_key_iv: "02c7d23c9323c739de398d7f3b1b60d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.571625 1504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:28.572357 1520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.572543 1504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:28.572645 1504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "8d58f87f9f644feb827add4fd5d98197"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "3ad7b44ffad03f7428e5ff382ce70b91"
server_key_iv: "02c7d23c9323c739de398d7f3b1b60d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.572726 1504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:28.582585 1504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:28.584254 1504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:28.584420 1504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:28.584645 1504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:28.585009 1504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:28.585048 1504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.585071 1504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:28.585119 1504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.591379 1504 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:36921
I20260810 21:03:28.591420 1646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:36921 every 8 connection(s)
I20260810 21:03:28.591697 1504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:28.599274 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1504
I20260810 21:03:28.599399 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:28.599608 15136 external_mini_cluster.cc:1428] Setting key 10fd9e65d0fa155e02cfd51206cd21bb
I20260810 21:03:28.600852 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.592279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54353 (local address 127.14.200.62:45917)
0810 21:03:28.592355 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.592358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.592532 (+ 174us) server_negotiation.cc:407] Connection header received
0810 21:03:28.593018 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.593020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.593031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.593091 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:28.593922 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.594470 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.595408 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.595595 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.595656 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.597700 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:28.597701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:28.597702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:28.597709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:28.599247 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.599773 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.599775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.599775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.599854 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.600140 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.600141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.600141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.600262 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:28.600346 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.600635 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.600721 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 21:03:28.601032 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.592232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45917 (local address 127.14.200.2:54353)
0810 21:03:28.592400 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.592429 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.592923 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.593137 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.593139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.593627 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:28.593798 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.593802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.594555 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.594555 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.595316 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.595318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.595681 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.595681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.595772 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.596187 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:28.596195 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:28.597607 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:28.599369 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.599377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.599378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.599689 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.599940 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.599941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.599941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.600057 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.600338 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:28.600355 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:28.600491 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.600610 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.600766 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260810 21:03:28.602430 1647 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45917
I20260810 21:03:28.602536 1647 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:28.602725 1647 heartbeater.cc:511] Master 127.14.200.62:45917 requested a full tablet report, sending...
I20260810 21:03:28.603108 1286 ts_manager.cc:194] Registered new tserver with Master: 8d58f87f9f644feb827add4fd5d98197 (127.14.200.2:36921)
I20260810 21:03:28.603463 1286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:54353
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:03:28.626935 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:28.704814 1652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:28.704993 1652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:28.705016 1652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:28.706044 1652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:28.706075 1652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:28.706099 1652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:28.707377 1652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:45917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:28.707554 1652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:28.707710 1652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:28.712504 1659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:28.712474 1658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.712494 1652 server_base.cc:1034] running on GCE node
W20260810 21:03:28.712455 1661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:28.713060 1652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:28.713284 1652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:28.714447 1652 hybrid_clock.cc:648] HybridClock initialized: now 1786395808714432 us; error 29 us; skew 500 ppm
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:28.716943 1652 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:03:28.717351 1652 webserver.cc:460] Webserver started at http://127.14.200.3:44067/ using document root <none> and password file <none>
I20260810 21:03:28.717595 1652 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:28.717664 1652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:28.717823 1652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:28.718767 1652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "f056568b322448d59f4ea4b048a7b3e2"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "6fbc78cf8caea615f14dff36d93f3977"
server_key_iv: "97fed18d62fef1b1f79befbac76463b4"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.719094 1652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "f056568b322448d59f4ea4b048a7b3e2"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "6fbc78cf8caea615f14dff36d93f3977"
server_key_iv: "97fed18d62fef1b1f79befbac76463b4"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.720359 1652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 21:03:28.721119 1669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.721283 1652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.721377 1652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "f056568b322448d59f4ea4b048a7b3e2"
format_stamp: "Formatted at 2026-08-10 21:03:28 on dist-test-slave-1x4g"
server_key: "6fbc78cf8caea615f14dff36d93f3977"
server_key_iv: "97fed18d62fef1b1f79befbac76463b4"
server_key_version: "encryptionkey@0"
I20260810 21:03:28.721459 1652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:28.732893 1652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:28.735023 1652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:28.735174 1652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:28.735381 1652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:28.735668 1652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:28.735723 1652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.735762 1652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:28.735790 1652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:28.741787 1652 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:38467
I20260810 21:03:28.741830 1795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:38467 every 8 connection(s)
I20260810 21:03:28.742117 1652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
Aug 10 21:03:28 dist-test-slave-1x4g krb5kdc[1239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395808, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:28.750494 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.742793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:55141 (local address 127.14.200.62:45917)
0810 21:03:28.742854 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.742856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.743083 (+ 227us) server_negotiation.cc:407] Connection header received
0810 21:03:28.743496 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.743497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.743506 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.743559 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:28.744365 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.744930 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.745895 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.746091 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.746146 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.748079 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:28.748080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:28.748080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:28.748087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:28.749372 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.749728 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.749728 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.749729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.749778 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.750007 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.750007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.750008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.750108 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:28.750158 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.750367 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.750422 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:03:28.750712 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.742771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:45917 (local address 127.14.200.3:55141)
0810 21:03:28.742939 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.742992 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.743410 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.743612 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.743614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.744128 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:28.744293 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.744297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.745003 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.745004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.745790 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.745792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.746165 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.746166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.746261 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.746639 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:28.746647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:28.748002 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:28.749444 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.749450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.749452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.749665 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.749837 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:28.749837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:28.749837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:28.749953 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:28.750163 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:28.750167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:28.750268 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.750357 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.750466 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260810 21:03:28.751293 1796 heartbeater.cc:347] Connected to a master server at 127.14.200.62:45917
I20260810 21:03:28.751394 1796 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:28.751585 1796 heartbeater.cc:511] Master 127.14.200.62:45917 requested a full tablet report, sending...
I20260810 21:03:28.751598 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1652
I20260810 21:03:28.751732 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:28.751950 15136 external_mini_cluster.cc:1428] Setting key 459652e5a6848c3fdb67d51cf315135d
I20260810 21:03:28.752024 1286 ts_manager.cc:194] Registered new tserver with Master: f056568b322448d59f4ea4b048a7b3e2 (127.14.200.3:38467)
I20260810 21:03:28.752504 1286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:55141
I20260810 21:03:28.752924 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:28.759604 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.754272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.14.200.62:45917)
0810 21:03:28.754309 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.754311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.754429 (+ 118us) server_negotiation.cc:407] Connection header received
0810 21:03:28.754456 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.754456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.754465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.754506 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:28.755008 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.755429 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.756123 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.756285 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.756332 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.757452 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:28.757453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:28.757453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:28.757458 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:28.758506 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.758835 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.758835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.758836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.758940 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:28.759112 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:28.759113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:28.759113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:28.759269 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:28.759342 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.759445 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.759505 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:03:28.761193 1285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:28.762701 1285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:28.768415 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:36921 (local address 127.0.0.1:51946)
0810 21:03:28.765046 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.765055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.765093 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.765476 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.765477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.765479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:28.765551 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.765552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766514 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.766515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.767382 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.767385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.768099 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.768187 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.768200 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.768290 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.768340 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260810 21:03:28.768415 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.14.200.2:36921)
0810 21:03:28.765211 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.765215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.765225 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:03:28.765264 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.765266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.765279 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.765366 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:28.765649 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766401 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.767473 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.768004 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768098 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.768153 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.768306 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.768349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":80,"threads_started":1}
I20260810 21:03:28.769219 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41237 (local address 127.0.0.1:47152)
0810 21:03:28.764962 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.764997 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.765074 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.765337 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.765341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.765343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:28.765409 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.765410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766921 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.766921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.768115 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.768118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.768987 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.769060 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.769063 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.769121 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.769158 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":58,"thread_start_us":61,"threads_started":1}
I20260810 21:03:28.769279 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.14.200.1:41237)
0810 21:03:28.765107 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.765111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.765136 (+ 25us) server_negotiation.cc:407] Connection header received
0810 21:03:28.765180 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.765181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.765193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.765499 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:28.765591 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766810 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768204 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.768911 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768976 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.769023 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.769146 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.769192 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260810 21:03:28.769367 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.14.200.3:38467)
0810 21:03:28.765015 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:28.765019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:28.765238 (+ 219us) server_negotiation.cc:407] Connection header received
0810 21:03:28.765276 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:28.765278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:28.765291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:28.765377 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:28.765689 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766786 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768210 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.768899 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.768960 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.769015 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:28.769251 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 21:03:28.769289 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260810 21:03:28.769464 1578 tablet_service.cc:1467] Processing CreateTablet for tablet ed7b75ffdc5543f387b87d5563e05954 (DEFAULT_TABLE table=test-table [id=a3128d988e2f42968a24cf72c24b98b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:28.769640 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:28.764697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:38467 (local address 127.0.0.1:45478)
0810 21:03:28.765164 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:28.765173 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:28.765221 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:28.765451 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:28.765452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:28.765454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:28.765496 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.765497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.766907 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.766908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.768116 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:28.768118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:28.769092 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:28.769093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:28.769159 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:28.769162 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:28.769526 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 21:03:28.769570 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":70,"thread_start_us":76,"threads_started":1}
I20260810 21:03:28.769912 1578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed7b75ffdc5543f387b87d5563e05954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:28.770027 1427 tablet_service.cc:1467] Processing CreateTablet for tablet ed7b75ffdc5543f387b87d5563e05954 (DEFAULT_TABLE table=test-table [id=a3128d988e2f42968a24cf72c24b98b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:28.770368 1427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed7b75ffdc5543f387b87d5563e05954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:28.770368 1724 tablet_service.cc:1467] Processing CreateTablet for tablet ed7b75ffdc5543f387b87d5563e05954 (DEFAULT_TABLE table=test-table [id=a3128d988e2f42968a24cf72c24b98b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:28.770682 1724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed7b75ffdc5543f387b87d5563e05954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:28.773053 1813 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Bootstrap starting.
I20260810 21:03:28.773701 1813 tablet_bootstrap.cc:654] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:28.773854 1815 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Bootstrap starting.
I20260810 21:03:28.774535 1815 tablet_bootstrap.cc:654] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:28.774540 1813 log.cc:824] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:28.775133 1815 log.cc:824] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:28.775358 1813 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: No bootstrap required, opened a new log
I20260810 21:03:28.775436 1813 ts_tablet_manager.cc:1397] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:28.775506 1816 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Bootstrap starting.
I20260810 21:03:28.775936 1815 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: No bootstrap required, opened a new log
I20260810 21:03:28.776036 1815 ts_tablet_manager.cc:1397] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:28.776481 1816 tablet_bootstrap.cc:654] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:28.777100 1816 log.cc:824] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:28.777701 1813 raft_consensus.cc:348] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.777877 1816 tablet_bootstrap.cc:492] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: No bootstrap required, opened a new log
I20260810 21:03:28.777896 1813 raft_consensus.cc:374] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:28.777931 1816 ts_tablet_manager.cc:1397] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:28.777943 1813 raft_consensus.cc:729] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 231cbccacdab4a61891413d8e78b50f4, State: Initialized, Role: FOLLOWER
I20260810 21:03:28.777884 1815 raft_consensus.cc:348] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.778082 1815 raft_consensus.cc:374] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:28.778100 1813 consensus_queue.cc:260] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.778156 1815 raft_consensus.cc:729] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f056568b322448d59f4ea4b048a7b3e2, State: Initialized, Role: FOLLOWER
I20260810 21:03:28.778313 1815 consensus_queue.cc:260] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.778373 1499 heartbeater.cc:503] Master 127.14.200.62:45917 was elected leader, sending a full tablet report...
I20260810 21:03:28.778650 1796 heartbeater.cc:503] Master 127.14.200.62:45917 was elected leader, sending a full tablet report...
I20260810 21:03:28.778666 1815 ts_tablet_manager.cc:1428] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 21:03:28.778360 1813 ts_tablet_manager.cc:1428] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:03:28.779917 1816 raft_consensus.cc:348] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.780069 1816 raft_consensus.cc:374] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:28.780114 1816 raft_consensus.cc:729] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d58f87f9f644feb827add4fd5d98197, State: Initialized, Role: FOLLOWER
I20260810 21:03:28.780217 1816 consensus_queue.cc:260] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:28.780494 1816 ts_tablet_manager.cc:1428] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:28.780515 1647 heartbeater.cc:503] Master 127.14.200.62:45917 was elected leader, sending a full tablet report...
W20260810 21:03:28.842247 1648 tablet.cc:2369] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:03:28.943926 1500 tablet.cc:2369] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:03:28.992723 1798 tablet.cc:2369] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:03:29.000154 1822 raft_consensus.cc:482] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:29.000336 1822 raft_consensus.cc:504] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:29.000754 1822 leader_election.cc:288] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f056568b322448d59f4ea4b048a7b3e2 (127.14.200.3:38467), 231cbccacdab4a61891413d8e78b50f4 (127.14.200.1:41237)
I20260810 21:03:29.004563 1649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.000954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:38467 (local address 127.14.200.2:50229)
0810 21:03:29.001007 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.001019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.001068 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.001278 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.001279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.001282 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.001361 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.001363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.002286 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.002286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.003497 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.003500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.004275 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.004276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.004367 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.004368 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.004424 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.004467 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":21}
I20260810 21:03:29.004570 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.001064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50229 (local address 127.14.200.3:38467)
0810 21:03:29.001117 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.001120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.001130 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:03:29.001166 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.001167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.001178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.001246 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.001448 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.002201 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.003611 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.004179 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.004269 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.004278 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.004453 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.004496 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 21:03:29.004570 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.001116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:57063 (local address 127.14.200.1:41237)
0810 21:03:29.001162 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.001164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.001327 (+ 163us) server_negotiation.cc:407] Connection header received
0810 21:03:29.001369 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.001370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.001382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.001465 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.001607 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.002288 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.003611 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.004185 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.004269 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.004278 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.004453 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.004496 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:03:29.004570 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.001027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41237 (local address 127.14.200.2:57063)
0810 21:03:29.001218 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.001246 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.001298 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.001482 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.001483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.001484 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.001532 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.001533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.002367 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.002367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.003497 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.003500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.004275 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.004276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.004362 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.004367 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.004436 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.004467 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":5,"thread_start_us":116,"threads_started":1}
I20260810 21:03:29.004976 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed7b75ffdc5543f387b87d5563e05954" candidate_uuid: "8d58f87f9f644feb827add4fd5d98197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f056568b322448d59f4ea4b048a7b3e2" is_pre_election: true
I20260810 21:03:29.005029 1450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed7b75ffdc5543f387b87d5563e05954" candidate_uuid: "8d58f87f9f644feb827add4fd5d98197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "231cbccacdab4a61891413d8e78b50f4" is_pre_election: true
I20260810 21:03:29.005167 1744 raft_consensus.cc:2454] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d58f87f9f644feb827add4fd5d98197 in term 0.
I20260810 21:03:29.005226 1450 raft_consensus.cc:2454] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d58f87f9f644feb827add4fd5d98197 in term 0.
I20260810 21:03:29.005470 1523 leader_election.cc:302] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d58f87f9f644feb827add4fd5d98197, f056568b322448d59f4ea4b048a7b3e2; no voters:
I20260810 21:03:29.005602 1822 raft_consensus.cc:2781] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:29.005667 1822 raft_consensus.cc:482] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:29.005702 1822 raft_consensus.cc:3037] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.006459 1822 raft_consensus.cc:504] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:29.006608 1822 leader_election.cc:288] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [CANDIDATE]: Term 1 election: Requested vote from peers f056568b322448d59f4ea4b048a7b3e2 (127.14.200.3:38467), 231cbccacdab4a61891413d8e78b50f4 (127.14.200.1:41237)
I20260810 21:03:29.006803 1450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed7b75ffdc5543f387b87d5563e05954" candidate_uuid: "8d58f87f9f644feb827add4fd5d98197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "231cbccacdab4a61891413d8e78b50f4"
I20260810 21:03:29.006840 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed7b75ffdc5543f387b87d5563e05954" candidate_uuid: "8d58f87f9f644feb827add4fd5d98197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f056568b322448d59f4ea4b048a7b3e2"
I20260810 21:03:29.006899 1450 raft_consensus.cc:3037] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.006925 1744 raft_consensus.cc:3037] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.007725 1744 raft_consensus.cc:2454] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d58f87f9f644feb827add4fd5d98197 in term 1.
I20260810 21:03:29.007783 1450 raft_consensus.cc:2454] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d58f87f9f644feb827add4fd5d98197 in term 1.
I20260810 21:03:29.007938 1523 leader_election.cc:302] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d58f87f9f644feb827add4fd5d98197, f056568b322448d59f4ea4b048a7b3e2; no voters:
I20260810 21:03:29.008020 1822 raft_consensus.cc:2781] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:29.008158 1822 raft_consensus.cc:686] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 1 LEADER]: Becoming Leader. State: Replica: 8d58f87f9f644feb827add4fd5d98197, State: Running, Role: LEADER
I20260810 21:03:29.008262 1822 consensus_queue.cc:237] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } }
I20260810 21:03:29.009195 1286 catalog_manager.cc:5495] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d58f87f9f644feb827add4fd5d98197 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "8d58f87f9f644feb827add4fd5d98197" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d58f87f9f644feb827add4fd5d98197" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 36921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 } health_report { overall_health: UNKNOWN } } }
I20260810 21:03:29.052695 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.050657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.14.200.2:36921)
0810 21:03:29.050724 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.050725 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.050733 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:29.050769 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.050769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.050788 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.050846 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:29.050949 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.051398 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.052080 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.052258 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.052306 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.052426 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:29.052518 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:29.052571 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.052603 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.052618 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 21:03:29.054656 1450 raft_consensus.cc:1261] T ed7b75ffdc5543f387b87d5563e05954 P 231cbccacdab4a61891413d8e78b50f4 [term 1 FOLLOWER]: Refusing update from remote peer 8d58f87f9f644feb827add4fd5d98197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:29.054656 1744 raft_consensus.cc:1261] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 1 FOLLOWER]: Refusing update from remote peer 8d58f87f9f644feb827add4fd5d98197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:29.055053 1824 consensus_queue.cc:1035] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "231cbccacdab4a61891413d8e78b50f4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:29.055190 1822 consensus_queue.cc:1035] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f056568b322448d59f4ea4b048a7b3e2" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 38467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:29.058108 1828 mvcc.cc:205] Tried to move back new op lower bound from 7317077233884688384 to 7317077233698570240. Current Snapshot: MvccSnapshot[applied={T|T < 7317077233884688384}]
I20260810 21:03:29.058614 1829 mvcc.cc:205] Tried to move back new op lower bound from 7317077233884688384 to 7317077233698570240. Current Snapshot: MvccSnapshot[applied={T|T < 7317077233884688384}]
I20260810 21:03:29.059701 1286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:29.059787 1286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:29.060556 1286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1cd76b366e74a5c879c4166afe4948b: Sending DeleteTablet for 3 replicas of tablet ed7b75ffdc5543f387b87d5563e05954
I20260810 21:03:29.060876 1578 tablet_service.cc:1514] Processing DeleteTablet for tablet ed7b75ffdc5543f387b87d5563e05954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:29 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:51946
I20260810 21:03:29.060932 1724 tablet_service.cc:1514] Processing DeleteTablet for tablet ed7b75ffdc5543f387b87d5563e05954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:29 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:45478
I20260810 21:03:29.061111 1837 tablet_replica.cc:330] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: stopping tablet replica
I20260810 21:03:29.061193 1838 tablet_replica.cc:330] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: stopping tablet replica
I20260810 21:03:29.061244 1837 raft_consensus.cc:2229] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:29.061302 1838 raft_consensus.cc:2229] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:29.061359 1837 raft_consensus.cc:2258] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:29.061415 1838 raft_consensus.cc:2258] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:29.061481 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1355
I20260810 21:03:29.063274 1837 ts_tablet_manager.cc:1905] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:29.064805 1837 ts_tablet_manager.cc:1918] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:29.064870 1837 log.cc:1196] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/ed7b75ffdc5543f387b87d5563e05954
I20260810 21:03:29.065146 1837 ts_tablet_manager.cc:1939] T ed7b75ffdc5543f387b87d5563e05954 P 8d58f87f9f644feb827add4fd5d98197: Deleting consensus metadata
I20260810 21:03:29.065932 1270 catalog_manager.cc:4841] TS 8d58f87f9f644feb827add4fd5d98197 (127.14.200.2:36921): tablet ed7b75ffdc5543f387b87d5563e05954 (table test-table [id=a3128d988e2f42968a24cf72c24b98b7]) successfully deleted
I20260810 21:03:29.066713 1838 ts_tablet_manager.cc:1905] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:29.068660 1838 ts_tablet_manager.cc:1918] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:29.068753 1838 log.cc:1196] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/ed7b75ffdc5543f387b87d5563e05954
I20260810 21:03:29.069041 1838 ts_tablet_manager.cc:1939] T ed7b75ffdc5543f387b87d5563e05954 P f056568b322448d59f4ea4b048a7b3e2: Deleting consensus metadata
I20260810 21:03:29.069607 1271 catalog_manager.cc:4841] TS f056568b322448d59f4ea4b048a7b3e2 (127.14.200.3:38467): tablet ed7b75ffdc5543f387b87d5563e05954 (table test-table [id=a3128d988e2f42968a24cf72c24b98b7]) successfully deleted
W20260810 21:03:29.071595 1272 connection.cc:535] client connection to 127.14.200.1:41237 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 21:03:29.071686 1272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 21:03:29.072218 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1504
I20260810 21:03:29.072242 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.072109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41237 (local address 127.0.0.1:47166)
0810 21:03:29.072157 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.072200 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 21:03:29.078146 1272 catalog_manager.cc:4584] TS 231cbccacdab4a61891413d8e78b50f4 (127.14.200.1:41237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed7b75ffdc5543f387b87d5563e05954: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41237: connect: Connection refused (error 111)
I20260810 21:03:29.082302 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1652
I20260810 21:03:29.089105 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1253
2026-08-10T21:03:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2117 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T21:03:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:29Z Disabled control of system clock
I20260810 21:03:29.119696 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39567
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:29.198073 1843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:29.198272 1843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:29.198292 1843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:29.199301 1843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:29.199332 1843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:29.199345 1843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:29.199357 1843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:29.199369 1843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:29.200625 1843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:29.200879 1843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:29.201133 1843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:29.205808 1851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.205821 1848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.205928 1843 server_base.cc:1034] running on GCE node
W20260810 21:03:29.205826 1849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.206351 1843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:29.206612 1843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:29.207748 1843 hybrid_clock.cc:648] HybridClock initialized: now 1786395809207739 us; error 27 us; skew 500 ppm
I20260810 21:03:29.208999 1843 webserver.cc:460] Webserver started at http://127.14.200.62:46673/ using document root <none> and password file <none>
I20260810 21:03:29.209246 1843 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:29.209290 1843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:29.209437 1843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:29.210501 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "4f974163ead44825b81b44104a221c03"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "4ef207f59334717ef6752ff21d6d3c08"
server_key_iv: "58da44a9be56cb2817d2af506167a806"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.210862 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "4f974163ead44825b81b44104a221c03"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "4ef207f59334717ef6752ff21d6d3c08"
server_key_iv: "58da44a9be56cb2817d2af506167a806"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.212098 1843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:29.212829 1859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.213006 1843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:29.213104 1843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "4f974163ead44825b81b44104a221c03"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "4ef207f59334717ef6752ff21d6d3c08"
server_key_iv: "58da44a9be56cb2817d2af506167a806"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.213181 1843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:29.223835 1843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:29.224120 1843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:29.224265 1843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:29.228468 1843 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:39567
I20260810 21:03:29.228514 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:39567 every 8 connection(s)
I20260810 21:03:29.228932 1843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:29.229920 1915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:29.232131 1915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: Bootstrap starting.
I20260810 21:03:29.232705 1915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:29.233105 1915 log.cc:824] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:29.233789 1915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: No bootstrap required, opened a new log
I20260810 21:03:29.234388 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1843
I20260810 21:03:29.234510 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:29.234709 15136 external_mini_cluster.cc:1428] Setting key 64d82ddfb91e5b54dc5f05d837471622
I20260810 21:03:29.236115 1915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } }
I20260810 21:03:29.236279 1915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:29.236315 1915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f974163ead44825b81b44104a221c03, State: Initialized, Role: FOLLOWER
I20260810 21:03:29.236488 1915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } }
I20260810 21:03:29.236586 1915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:29.236632 1915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:29.236680 1915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.237416 1915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } }
I20260810 21:03:29.237528 1915 leader_election.cc:302] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f974163ead44825b81b44104a221c03; no voters:
I20260810 21:03:29.237682 1915 leader_election.cc:288] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:29.237759 1920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:29.237946 1920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [term 1 LEADER]: Becoming Leader. State: Replica: 4f974163ead44825b81b44104a221c03, State: Running, Role: LEADER
I20260810 21:03:29.237923 1915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:29.238131 1920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } }
I20260810 21:03:29.238550 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f974163ead44825b81b44104a221c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } } }
I20260810 21:03:29.238574 1922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f974163ead44825b81b44104a221c03. Latest consensus state: current_term: 1 leader_uuid: "4f974163ead44825b81b44104a221c03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f974163ead44825b81b44104a221c03" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39567 } } }
I20260810 21:03:29.238727 1922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:29.238718 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:29.238988 1929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:29.239593 1929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:29.241531 1929 catalog_manager.cc:1347] Generated new cluster ID: 2db0081a41684e169a71d649fb97f252
I20260810 21:03:29.241596 1929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:29.241757 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.235165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.14.200.62:39567)
0810 21:03:29.235378 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.235386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.235417 (+ 31us) server_negotiation.cc:407] Connection header received
0810 21:03:29.236074 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.236084 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.236161 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.236308 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:29.236621 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.238344 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.239518 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.240140 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.240260 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.240995 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:29.240998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:29.240998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:29.241209 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:29.241248 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:29.241261 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:29.241322 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.241449 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.241529 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260810 21:03:29.258479 1929 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:29.259158 1929 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:29.264725 1929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: Generated new TSK 0
I20260810 21:03:29.264909 1929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:29.268424 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--builtin_ntp_servers=127.14.200.20:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:29.351150 1941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:29.351338 1941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:29.351359 1941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:29.352447 1941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:29.352485 1941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:29.352509 1941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:29.353881 1941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:29.354110 1941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:29.354409 1941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:29.360056 1941 server_base.cc:1034] running on GCE node
W20260810 21:03:29.360038 1949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.360026 1951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.360010 1947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.360473 1941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:29.360721 1941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:29.361901 1941 hybrid_clock.cc:648] HybridClock initialized: now 1786395809361878 us; error 36 us; skew 500 ppm
I20260810 21:03:29.363044 1941 webserver.cc:460] Webserver started at http://127.14.200.1:46767/ using document root <none> and password file <none>
I20260810 21:03:29.363272 1941 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:29.363336 1941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:29.363476 1941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:29.364537 1941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "f16e8e94b66140bb9645e27c8db6503e"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "d273e2d981672213af477c7ac2216f1d"
server_key_iv: "5e7f034b323f5b9fa2346f5a4e78fd7a"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.364913 1941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "f16e8e94b66140bb9645e27c8db6503e"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "d273e2d981672213af477c7ac2216f1d"
server_key_iv: "5e7f034b323f5b9fa2346f5a4e78fd7a"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.366163 1941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:03:29.367024 1957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.367218 1941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:29.367314 1941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "f16e8e94b66140bb9645e27c8db6503e"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "d273e2d981672213af477c7ac2216f1d"
server_key_iv: "5e7f034b323f5b9fa2346f5a4e78fd7a"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.367401 1941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:29.375377 1941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:29.375619 1941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:29.375751 1941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:29.375945 1941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:29.376245 1941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:29.376302 1941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.376338 1941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:29.376389 1941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.381999 1941 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:44779
I20260810 21:03:29.382102 2077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:44779 every 8 connection(s)
I20260810 21:03:29.382365 1941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:29.384092 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 1941
I20260810 21:03:29.384227 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:29.384487 15136 external_mini_cluster.cc:1428] Setting key f859c8f3ab4d0839856d5650e80b4537
I20260810 21:03:29.385605 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--builtin_ntp_servers=127.14.200.20:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 21:03:29.388159 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.383069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:36963 (local address 127.14.200.62:39567)
0810 21:03:29.383140 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.383142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.383326 (+ 184us) server_negotiation.cc:407] Connection header received
0810 21:03:29.383758 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.383759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.383769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.383813 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:29.384691 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.385349 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.386768 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.386950 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.387002 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.387915 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:29.387916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:29.387917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:29.387921 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:29.387935 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:29.387939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:29.387986 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.388059 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.388085 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 21:03:29.388362 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.383026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39567 (local address 127.14.200.1:36963)
0810 21:03:29.383220 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.383250 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.383682 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.383867 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.383877 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.383884 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:29.384595 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.384603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.385446 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.385447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.386683 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.386687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.387040 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.387041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.387197 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.387751 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:29.387752 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:29.387783 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:29.387804 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:29.387986 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:29.387991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:29.387994 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.388060 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.388154 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260810 21:03:29.388909 2078 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39567
I20260810 21:03:29.389024 2078 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:29.389189 2078 heartbeater.cc:511] Master 127.14.200.62:39567 requested a full tablet report, sending...
I20260810 21:03:29.389663 1878 ts_manager.cc:194] Registered new tserver with Master: f16e8e94b66140bb9645e27c8db6503e (127.14.200.1:44779)
I20260810 21:03:29.390273 1878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.1:36963
W20260810 21:03:29.467958 2081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:29.468163 2081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:29.468199 2081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:29.469333 2081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:29.469388 2081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:29.469427 2081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:29.471004 2081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:29.471268 2081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:29.471545 2081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:29.476294 2081 server_base.cc:1034] running on GCE node
W20260810 21:03:29.476279 2090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.476261 2086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.476269 2092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.476734 2081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:29.477036 2081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:29.478194 2081 hybrid_clock.cc:648] HybridClock initialized: now 1786395809478187 us; error 43 us; skew 500 ppm
I20260810 21:03:29.479280 2081 webserver.cc:460] Webserver started at http://127.14.200.2:36803/ using document root <none> and password file <none>
I20260810 21:03:29.479511 2081 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:29.479566 2081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:29.479697 2081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:29.480703 2081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "135f60dd94174fddb3e264f7dc4c1edb"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "18a4d8eccaa148b308ee6eb22d0d50fe"
server_key_iv: "f2991d93f8c715e57a945466d0f4aa86"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.481173 2081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "135f60dd94174fddb3e264f7dc4c1edb"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "18a4d8eccaa148b308ee6eb22d0d50fe"
server_key_iv: "f2991d93f8c715e57a945466d0f4aa86"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.482316 2081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:03:29.482960 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.483127 2081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:29.483215 2081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "135f60dd94174fddb3e264f7dc4c1edb"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "18a4d8eccaa148b308ee6eb22d0d50fe"
server_key_iv: "f2991d93f8c715e57a945466d0f4aa86"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.483292 2081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:29.500862 2081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:29.501138 2081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:29.501283 2081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:29.501492 2081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:29.501796 2081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:29.501847 2081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.501893 2081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:29.501921 2081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.507601 2081 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:40601
I20260810 21:03:29.507638 2219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:40601 every 8 connection(s)
I20260810 21:03:29.507965 2081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:03:29.511648 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2081
I20260810 21:03:29.511770 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:29.511983 15136 external_mini_cluster.cc:1428] Setting key 328ef2c6e08b629922c4449807277ad4
I20260810 21:03:29.512960 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--builtin_ntp_servers=127.14.200.20:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 21:03:29.513401 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.508666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:37123 (local address 127.14.200.62:39567)
0810 21:03:29.508725 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.508728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.508947 (+ 219us) server_negotiation.cc:407] Connection header received
0810 21:03:29.509448 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.509450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.509463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.509524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:29.510212 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.510713 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.511781 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.512098 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.512179 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.513019 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:29.513021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:29.513021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:29.513026 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:29.513040 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:29.513045 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:29.513091 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.513230 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.513259 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:03:29.513506 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.508631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39567 (local address 127.14.200.2:37123)
0810 21:03:29.508815 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.508855 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.509342 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.509593 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.509598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.509607 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:29.510112 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.510117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.510813 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.510814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.511626 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.511628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.512195 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.512196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.512341 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.512885 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:29.512885 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:29.512911 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:29.512932 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:29.513117 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:29.513123 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:29.513126 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.513214 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.513327 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260810 21:03:29.514087 2220 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39567
I20260810 21:03:29.514185 2220 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:29.514361 2220 heartbeater.cc:511] Master 127.14.200.62:39567 requested a full tablet report, sending...
I20260810 21:03:29.514711 1878 ts_manager.cc:194] Registered new tserver with Master: 135f60dd94174fddb3e264f7dc4c1edb (127.14.200.2:40601)
I20260810 21:03:29.515105 1878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.2:37123
W20260810 21:03:29.593319 2223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:29.593534 2223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:29.593555 2223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:29.594738 2223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:29.594771 2223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:29.594794 2223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:29.595994 2223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:29.596160 2223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:29.596381 2223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:29.601233 2229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:29.601243 2230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.601334 2223 server_base.cc:1034] running on GCE node
W20260810 21:03:29.601315 2233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:29.601677 2223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:29.601914 2223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:29.603080 2223 hybrid_clock.cc:648] HybridClock initialized: now 1786395809603064 us; error 27 us; skew 500 ppm
I20260810 21:03:29.604303 2223 webserver.cc:460] Webserver started at http://127.14.200.3:42689/ using document root <none> and password file <none>
I20260810 21:03:29.604534 2223 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:29.604604 2223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:29.604754 2223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:29.605796 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "1d5e836a5c274abda6a1a94d806f534b"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "e9de9179b7e9d9996f37d967f217c764"
server_key_iv: "f7cb1356101fd441f88cdc3c831409d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.606145 2223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5e836a5c274abda6a1a94d806f534b"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "e9de9179b7e9d9996f37d967f217c764"
server_key_iv: "f7cb1356101fd441f88cdc3c831409d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.607378 2223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 21:03:29.608047 2240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.608210 2223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:29.608304 2223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "1d5e836a5c274abda6a1a94d806f534b"
format_stamp: "Formatted at 2026-08-10 21:03:29 on dist-test-slave-1x4g"
server_key: "e9de9179b7e9d9996f37d967f217c764"
server_key_iv: "f7cb1356101fd441f88cdc3c831409d6"
server_key_version: "encryptionkey@0"
I20260810 21:03:29.608381 2223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:29.626621 2223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:29.626876 2223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:29.627044 2223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:29.627259 2223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:29.627569 2223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:29.627619 2223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.627650 2223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:29.627673 2223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:29.633304 2223 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:40837
I20260810 21:03:29.633343 2364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:40837 every 8 connection(s)
I20260810 21:03:29.633754 2223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:03:29.638860 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.634482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:59759 (local address 127.14.200.62:39567)
0810 21:03:29.634530 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.634532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.634765 (+ 233us) server_negotiation.cc:407] Connection header received
0810 21:03:29.635262 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.635264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.635277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.635339 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:29.636006 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.636496 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.637571 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.637799 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.637892 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.638655 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:29.638656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:29.638656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:29.638660 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:29.638679 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:29.638684 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:29.638747 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.638788 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.638808 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:03:29.639081 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.634449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39567 (local address 127.14.200.3:59759)
0810 21:03:29.634651 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.634680 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.635159 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.635404 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.635408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.635417 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:29.635912 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.635917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.636602 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.636604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.637490 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.637492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.637902 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.637903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.638034 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.638521 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:29.638521 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:29.638547 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:29.638567 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:29.638743 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:29.638745 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:29.638747 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.638790 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.638861 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260810 21:03:29.639093 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2223
I20260810 21:03:29.639209 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:29.639452 15136 external_mini_cluster.cc:1428] Setting key c3f4bb539dc3f3b3451df34dd83ded4e
I20260810 21:03:29.639907 2365 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39567
I20260810 21:03:29.640046 2365 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:29.640233 2365 heartbeater.cc:511] Master 127.14.200.62:39567 requested a full tablet report, sending...
I20260810 21:03:29.640640 1878 ts_manager.cc:194] Registered new tserver with Master: 1d5e836a5c274abda6a1a94d806f534b (127.14.200.3:40837)
I20260810 21:03:29.641054 1878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.3:59759
I20260810 21:03:29.651113 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:29.654985 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.652483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.14.200.62:39567)
0810 21:03:29.652560 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.652562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.652629 (+ 67us) server_negotiation.cc:407] Connection header received
0810 21:03:29.652667 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.652668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.652676 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.652720 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:29.652927 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.653372 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.654033 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.654198 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.654245 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.654770 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:29.654771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:29.654772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:29.654775 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:29.654787 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:29.654790 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:29.654830 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.654884 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.654902 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 21:03:29.656311 1878 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:29.657913 1878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:29.663867 2373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.659911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:40837 (local address 127.0.0.1:56030)
0810 21:03:29.660156 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.660166 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.660215 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.660806 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.660810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.660812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.660873 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.660875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.661696 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.661697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.662639 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.662642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.663593 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.663667 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.663678 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.663744 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.663803 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":62,"thread_start_us":65,"threads_started":1}
I20260810 21:03:29.663904 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.660032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.14.200.3:40837)
0810 21:03:29.660572 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.660575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.660583 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:29.660612 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.660613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.660624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.660842 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.660961 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.661612 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.662745 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.663414 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663459 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.663531 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.663778 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.663821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":92,"threads_started":1}
I20260810 21:03:29.664258 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.659911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40601 (local address 127.0.0.1:54482)
0810 21:03:29.660100 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.660137 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.660196 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.660807 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.660813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.660817 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.660880 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.660882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.661744 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.661745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.662981 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.662984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.664054 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.664055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.664110 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.664113 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.664154 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.664193 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260810 21:03:29.664498 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.660316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.14.200.2:40601)
0810 21:03:29.660518 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.660523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.660534 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:03:29.660576 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.660578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.660594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.660679 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.660992 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.661637 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663067 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.663754 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663817 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.663895 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.664376 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.664414 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260810 21:03:29.664518 2294 tablet_service.cc:1467] Processing CreateTablet for tablet 28514f2b998942a9930fc95508cb2e8c (DEFAULT_TABLE table=test-table [id=1999968eda354c96a45f333748f47f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:29.664932 2294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28514f2b998942a9930fc95508cb2e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:29.665159 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.660572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.14.200.1:44779)
0810 21:03:29.661762 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.661766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.661775 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:29.661810 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.661811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.661825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.661917 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.662034 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.663070 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663882 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.664688 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.664752 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.664874 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.665034 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.665081 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1149,"thread_start_us":118,"threads_started":1}
I20260810 21:03:29.665170 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.660150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:44779 (local address 127.0.0.1:56654)
0810 21:03:29.660316 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.660325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.660368 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.661930 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.661931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.661933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.661967 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.661969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.663151 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.663152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.663793 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.663795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.664829 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.664830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.664926 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.664938 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.664994 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.665108 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260810 21:03:29.666030 2008 tablet_service.cc:1467] Processing CreateTablet for tablet 28514f2b998942a9930fc95508cb2e8c (DEFAULT_TABLE table=test-table [id=1999968eda354c96a45f333748f47f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:29.666350 2008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28514f2b998942a9930fc95508cb2e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:29.667611 2380 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Bootstrap starting.
I20260810 21:03:29.668149 2150 tablet_service.cc:1467] Processing CreateTablet for tablet 28514f2b998942a9930fc95508cb2e8c (DEFAULT_TABLE table=test-table [id=1999968eda354c96a45f333748f47f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:29.668504 2150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28514f2b998942a9930fc95508cb2e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:29.668592 2380 tablet_bootstrap.cc:654] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:29.668916 2380 log.cc:824] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:29.669075 2381 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: Bootstrap starting.
I20260810 21:03:29.669623 2380 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: No bootstrap required, opened a new log
I20260810 21:03:29.669699 2380 ts_tablet_manager.cc:1397] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:03:29.669759 2381 tablet_bootstrap.cc:654] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:29.670058 2381 log.cc:824] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:29.671983 2384 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Bootstrap starting.
I20260810 21:03:29.672425 2380 raft_consensus.cc:348] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.672515 2381 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: No bootstrap required, opened a new log
I20260810 21:03:29.672573 2381 ts_tablet_manager.cc:1397] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 21:03:29.672574 2380 raft_consensus.cc:374] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:29.672613 2380 raft_consensus.cc:729] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5e836a5c274abda6a1a94d806f534b, State: Initialized, Role: FOLLOWER
I20260810 21:03:29.672616 2384 tablet_bootstrap.cc:654] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:29.672751 2380 consensus_queue.cc:260] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.672940 2384 log.cc:824] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:29.673038 2380 ts_tablet_manager.cc:1428] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 21:03:29.673168 2365 heartbeater.cc:503] Master 127.14.200.62:39567 was elected leader, sending a full tablet report...
I20260810 21:03:29.673681 2384 tablet_bootstrap.cc:492] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: No bootstrap required, opened a new log
I20260810 21:03:29.673744 2384 ts_tablet_manager.cc:1397] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:03:29.675022 2381 raft_consensus.cc:348] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.675166 2381 raft_consensus.cc:374] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:29.675197 2381 raft_consensus.cc:729] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16e8e94b66140bb9645e27c8db6503e, State: Initialized, Role: FOLLOWER
I20260810 21:03:29.675304 2381 consensus_queue.cc:260] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.675551 2381 ts_tablet_manager.cc:1428] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:03:29.675604 2078 heartbeater.cc:503] Master 127.14.200.62:39567 was elected leader, sending a full tablet report...
I20260810 21:03:29.676033 2384 raft_consensus.cc:348] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.676182 2384 raft_consensus.cc:374] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:29.676236 2384 raft_consensus.cc:729] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135f60dd94174fddb3e264f7dc4c1edb, State: Initialized, Role: FOLLOWER
I20260810 21:03:29.676345 2384 consensus_queue.cc:260] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.676604 2384 ts_tablet_manager.cc:1428] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:03:29.676694 2220 heartbeater.cc:503] Master 127.14.200.62:39567 was elected leader, sending a full tablet report...
I20260810 21:03:29.755640 2387 raft_consensus.cc:482] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:29.755757 2387 raft_consensus.cc:504] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.756166 2387 leader_election.cc:288] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 135f60dd94174fddb3e264f7dc4c1edb (127.14.200.2:40601), 1d5e836a5c274abda6a1a94d806f534b (127.14.200.3:40837)
W20260810 21:03:29.758509 2221 tablet.cc:2369] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:03:29.758828 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.756327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:40837 (local address 127.14.200.1:39265)
0810 21:03:29.756381 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.756393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.756434 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.756584 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.756584 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.756586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.756696 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.756698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.757347 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.757348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.758075 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.758077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.758607 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.758676 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.758680 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.758722 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.758765 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 21:03:29.758844 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.756395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:39265 (local address 127.14.200.3:40837)
0810 21:03:29.756435 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.756437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.756445 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:29.756498 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.756499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.756507 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.756580 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.756768 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.757262 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758180 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.758531 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758607 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.758615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.758753 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.758789 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:03:29.759169 2314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28514f2b998942a9930fc95508cb2e8c" candidate_uuid: "f16e8e94b66140bb9645e27c8db6503e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5e836a5c274abda6a1a94d806f534b" is_pre_election: true
I20260810 21:03:29.759227 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.756406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:33905 (local address 127.14.200.2:40601)
0810 21:03:29.756453 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.756455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.756603 (+ 148us) server_negotiation.cc:407] Connection header received
0810 21:03:29.756700 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.756701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.756718 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.756779 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:29.756973 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.757412 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758331 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.758844 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758931 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.758939 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.759147 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.759185 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:03:29.759235 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.756336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:40601 (local address 127.14.200.1:33905)
0810 21:03:29.756514 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.756530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:29.756610 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:29.756801 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:29.756802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:29.756803 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:29.756883 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.756884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.757501 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.757502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.758243 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:29.758244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.758991 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.758992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:29.759072 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.759075 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:29.759130 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 21:03:29.759164 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":14,"thread_start_us":135,"threads_started":1}
I20260810 21:03:29.759325 2314 raft_consensus.cc:2454] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16e8e94b66140bb9645e27c8db6503e in term 0.
I20260810 21:03:29.759543 1960 leader_election.cc:302] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5e836a5c274abda6a1a94d806f534b, f16e8e94b66140bb9645e27c8db6503e; no voters:
I20260810 21:03:29.759653 2387 raft_consensus.cc:2781] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:29.759706 2387 raft_consensus.cc:482] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:29.759728 2387 raft_consensus.cc:3037] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.759758 2171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28514f2b998942a9930fc95508cb2e8c" candidate_uuid: "f16e8e94b66140bb9645e27c8db6503e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135f60dd94174fddb3e264f7dc4c1edb" is_pre_election: true
I20260810 21:03:29.759902 2171 raft_consensus.cc:2454] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16e8e94b66140bb9645e27c8db6503e in term 0.
I20260810 21:03:29.760612 2387 raft_consensus.cc:504] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.760754 2387 leader_election.cc:288] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [CANDIDATE]: Term 1 election: Requested vote from peers 135f60dd94174fddb3e264f7dc4c1edb (127.14.200.2:40601), 1d5e836a5c274abda6a1a94d806f534b (127.14.200.3:40837)
I20260810 21:03:29.760907 2314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28514f2b998942a9930fc95508cb2e8c" candidate_uuid: "f16e8e94b66140bb9645e27c8db6503e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5e836a5c274abda6a1a94d806f534b"
I20260810 21:03:29.760923 2171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28514f2b998942a9930fc95508cb2e8c" candidate_uuid: "f16e8e94b66140bb9645e27c8db6503e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "135f60dd94174fddb3e264f7dc4c1edb"
I20260810 21:03:29.760990 2314 raft_consensus.cc:3037] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.761010 2171 raft_consensus.cc:3037] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:29.761736 2314 raft_consensus.cc:2454] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16e8e94b66140bb9645e27c8db6503e in term 1.
I20260810 21:03:29.761902 1960 leader_election.cc:302] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5e836a5c274abda6a1a94d806f534b, f16e8e94b66140bb9645e27c8db6503e; no voters:
I20260810 21:03:29.761940 2171 raft_consensus.cc:2454] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16e8e94b66140bb9645e27c8db6503e in term 1.
I20260810 21:03:29.761993 2387 raft_consensus.cc:2781] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:29.762157 2387 raft_consensus.cc:686] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 1 LEADER]: Becoming Leader. State: Replica: f16e8e94b66140bb9645e27c8db6503e, State: Running, Role: LEADER
I20260810 21:03:29.762246 2387 consensus_queue.cc:237] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } }
I20260810 21:03:29.763027 1878 catalog_manager.cc:5495] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e reported cstate change: term changed from 0 to 1, leader changed from <none> to f16e8e94b66140bb9645e27c8db6503e (127.14.200.1). New cstate: current_term: 1 leader_uuid: "f16e8e94b66140bb9645e27c8db6503e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16e8e94b66140bb9645e27c8db6503e" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 44779 } health_report { overall_health: HEALTHY } } }
I20260810 21:03:29.778294 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.776173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.14.200.1:44779)
0810 21:03:29.776261 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:29.776263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:29.776270 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:03:29.776298 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:29.776298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:29.776316 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:29.776373 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:29.776469 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.776970 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.777648 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:29.777835 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:29.777880 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:29.778006 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:29.778091 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:29.778147 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:29.778181 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 21:03:29.778197 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:29.779908 2171 raft_consensus.cc:1261] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 1 FOLLOWER]: Refusing update from remote peer f16e8e94b66140bb9645e27c8db6503e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:29.779917 2314 raft_consensus.cc:1261] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 1 FOLLOWER]: Refusing update from remote peer f16e8e94b66140bb9645e27c8db6503e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:29.780265 2387 consensus_queue.cc:1035] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5e836a5c274abda6a1a94d806f534b" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:29.780443 2390 consensus_queue.cc:1035] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [LEADER]: Connected to new peer: Peer: permanent_uuid: "135f60dd94174fddb3e264f7dc4c1edb" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 40601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:29.782959 2395 mvcc.cc:205] Tried to move back new op lower bound from 7317077236856111104 to 7317077236786606080. Current Snapshot: MvccSnapshot[applied={T|T < 7317077236856111104}]
I20260810 21:03:29.783881 2396 mvcc.cc:205] Tried to move back new op lower bound from 7317077236856111104 to 7317077236786606080. Current Snapshot: MvccSnapshot[applied={T|T < 7317077236856111104 or (T in {7317077236856111104})}]
I20260810 21:03:29.785050 1878 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:29.785141 1878 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:29.785796 1878 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f974163ead44825b81b44104a221c03: Sending DeleteTablet for 3 replicas of tablet 28514f2b998942a9930fc95508cb2e8c
I20260810 21:03:29.786113 2150 tablet_service.cc:1514] Processing DeleteTablet for tablet 28514f2b998942a9930fc95508cb2e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:29 UTC) from {username='slave'} at 127.0.0.1:54482
I20260810 21:03:29.786134 2008 tablet_service.cc:1514] Processing DeleteTablet for tablet 28514f2b998942a9930fc95508cb2e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:29 UTC) from {username='slave'} at 127.0.0.1:56654
I20260810 21:03:29.786350 2404 tablet_replica.cc:330] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: stopping tablet replica
I20260810 21:03:29.786331 2294 tablet_service.cc:1514] Processing DeleteTablet for tablet 28514f2b998942a9930fc95508cb2e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:29 UTC) from {username='slave'} at 127.0.0.1:56030
I20260810 21:03:29.786445 2404 raft_consensus.cc:2229] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:29.786554 2404 raft_consensus.cc:2258] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:29.786703 2405 tablet_replica.cc:330] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e: stopping tablet replica
I20260810 21:03:29.786777 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1941
I20260810 21:03:29.786796 2405 raft_consensus.cc:2229] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:29.786873 2405 raft_consensus.cc:2258] T 28514f2b998942a9930fc95508cb2e8c P f16e8e94b66140bb9645e27c8db6503e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:29.787068 2406 tablet_replica.cc:330] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: stopping tablet replica
I20260810 21:03:29.787150 2406 raft_consensus.cc:2229] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:29.787230 2406 raft_consensus.cc:2258] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:29.787773 2406 ts_tablet_manager.cc:1905] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:29.789655 2404 ts_tablet_manager.cc:1905] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:29.790053 2406 ts_tablet_manager.cc:1918] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:29.790125 2406 log.cc:1196] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/28514f2b998942a9930fc95508cb2e8c
I20260810 21:03:29.790378 2406 ts_tablet_manager.cc:1939] T 28514f2b998942a9930fc95508cb2e8c P 1d5e836a5c274abda6a1a94d806f534b: Deleting consensus metadata
I20260810 21:03:29.791024 1862 catalog_manager.cc:4841] TS 1d5e836a5c274abda6a1a94d806f534b (127.14.200.3:40837): tablet 28514f2b998942a9930fc95508cb2e8c (table test-table [id=1999968eda354c96a45f333748f47f70]) successfully deleted
I20260810 21:03:29.791484 2404 ts_tablet_manager.cc:1918] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:29.791548 2404 log.cc:1196] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/28514f2b998942a9930fc95508cb2e8c
I20260810 21:03:29.791774 2404 ts_tablet_manager.cc:1939] T 28514f2b998942a9930fc95508cb2e8c P 135f60dd94174fddb3e264f7dc4c1edb: Deleting consensus metadata
I20260810 21:03:29.792158 1861 catalog_manager.cc:4841] TS 135f60dd94174fddb3e264f7dc4c1edb (127.14.200.2:40601): tablet 28514f2b998942a9930fc95508cb2e8c (table test-table [id=1999968eda354c96a45f333748f47f70]) successfully deleted
W20260810 21:03:29.796468 1860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:44779 (error 108)
I20260810 21:03:29.796731 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2081
I20260810 21:03:29.797022 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:29.796901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:44779 (local address 127.0.0.1:56682)
0810 21:03:29.796949 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:29.796987 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:44779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:03:29.797122 1860 catalog_manager.cc:4584] TS f16e8e94b66140bb9645e27c8db6503e (127.14.200.1:44779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28514f2b998942a9930fc95508cb2e8c: Network error: Client connection negotiation failed: client connection to 127.14.200.1:44779: connect: Connection refused (error 111)
I20260810 21:03:29.804297 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2223
I20260810 21:03:29.809864 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 1843
2026-08-10T21:03:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (719 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:29 dist-test-slave-1x4g krb5kdc[2411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:29 dist-test-slave-1x4g krb5kdc[2411](info): set up 2 sockets
Aug 10 21:03:29 dist-test-slave-1x4g krb5kdc[2411](info): commencing operation
krb5kdc: starting...
W20260810 21:03:30.893334 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:30 dist-test-slave-1x4g krb5kdc[2411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:31.023331 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41639
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:31.101137 2427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:31.101369 2427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:31.101408 2427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:31.102499 2427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:31.102571 2427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:31.102593 2427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:31.102613 2427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:31.102643 2427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:31.104506 2427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:31.104930 2427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:31.105237 2427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:31.111348 2432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.111768 2434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.111819 2436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.112579 2427 server_base.cc:1034] running on GCE node
I20260810 21:03:31.112881 2427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:31.113190 2427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:31.114406 2427 hybrid_clock.cc:648] HybridClock initialized: now 1786395811114393 us; error 35 us; skew 500 ppm
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:31.116847 2427 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:31.117339 2427 webserver.cc:460] Webserver started at http://127.14.200.62:37899/ using document root <none> and password file <none>
I20260810 21:03:31.117568 2427 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:31.117635 2427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:31.117790 2427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:31.118814 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "93947dc3202340c8b267be49789c823b"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "73c8c944283da948506c84818a2ceb6c"
server_key_iv: "33a25fc95e8d9ea9fa208de5d3a214f8"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.119179 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "93947dc3202340c8b267be49789c823b"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "73c8c944283da948506c84818a2ceb6c"
server_key_iv: "33a25fc95e8d9ea9fa208de5d3a214f8"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.120494 2427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:03:31.121272 2443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.121441 2427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.121541 2427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "93947dc3202340c8b267be49789c823b"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "73c8c944283da948506c84818a2ceb6c"
server_key_iv: "33a25fc95e8d9ea9fa208de5d3a214f8"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.121626 2427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:31.128034 2427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:31.130303 2427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:31.130466 2427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:31.134447 2427 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41639
I20260810 21:03:31.134492 2498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41639 every 8 connection(s)
I20260810 21:03:31.134851 2427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:31.135706 2499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:31.137424 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2427
I20260810 21:03:31.137562 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:31.137768 15136 external_mini_cluster.cc:1428] Setting key 59e2e36e021783627a46aeaba006c146
I20260810 21:03:31.138319 2499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: Bootstrap starting.
I20260810 21:03:31.138942 2499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:31.139245 2499 log.cc:824] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:31.139891 2499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: No bootstrap required, opened a new log
I20260810 21:03:31.141750 2499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } }
I20260810 21:03:31.141901 2499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:31.141959 2499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93947dc3202340c8b267be49789c823b, State: Initialized, Role: FOLLOWER
I20260810 21:03:31.142127 2499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } }
I20260810 21:03:31.142218 2499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:31.142262 2499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:31.142307 2499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:31.142845 2499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } }
I20260810 21:03:31.142961 2499 leader_election.cc:302] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93947dc3202340c8b267be49789c823b; no voters:
I20260810 21:03:31.143127 2499 leader_election.cc:288] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:31.143148 2504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:31.143256 2504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [term 1 LEADER]: Becoming Leader. State: Replica: 93947dc3202340c8b267be49789c823b, State: Running, Role: LEADER
I20260810 21:03:31.143385 2499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:31.143393 2504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } }
I20260810 21:03:31.143715 2504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93947dc3202340c8b267be49789c823b. Latest consensus state: current_term: 1 leader_uuid: "93947dc3202340c8b267be49789c823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } } }
I20260810 21:03:31.143788 2504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:31.144050 2510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:31.144670 2510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:31.145515 2506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93947dc3202340c8b267be49789c823b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93947dc3202340c8b267be49789c823b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41639 } } }
I20260810 21:03:31.145598 2506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:31.146556 2510 catalog_manager.cc:1347] Generated new cluster ID: 6e94353338824208a886c5ce575c6220
I20260810 21:03:31.146625 2510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:31.148805 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.138279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.14.200.62:41639)
0810 21:03:31.138447 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.138454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.138477 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:03:31.138888 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.138894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.138944 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.139084 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:31.139775 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.140700 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.141497 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.141900 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.142026 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.144977 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:31.144981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:31.144981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:31.144992 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:31.147196 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.147575 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.147579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.147579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.147688 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.147887 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.147887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.147888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.148185 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:31.148337 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.148431 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.148579 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260810 21:03:31.166587 2510 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:31.167315 2510 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:31.175334 2510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: Generated new TSK 0
I20260810 21:03:31.175499 2510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:03:31.204876 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:31.285105 2529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:31.285301 2529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:31.285322 2529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:31.286294 2529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:31.286324 2529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:31.286347 2529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:31.287632 2529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:31.287793 2529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:31.287959 2529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:31.292608 2537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.292711 2529 server_base.cc:1034] running on GCE node
W20260810 21:03:31.292726 2539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.292608 2536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.293071 2529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:31.293313 2529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:31.294476 2529 hybrid_clock.cc:648] HybridClock initialized: now 1786395811294463 us; error 26 us; skew 500 ppm
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:31.297048 2529 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:03:31.297405 2529 webserver.cc:460] Webserver started at http://127.14.200.1:41911/ using document root <none> and password file <none>
I20260810 21:03:31.297574 2529 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:31.297613 2529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:31.297698 2529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:31.298646 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "72cb131c83b749d486fb6435b22d60b4"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "12bd7325b471b5b83199f1a711fe5cdc"
server_key_iv: "daff92f7e35a7201b3149e80b5eef100"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.298955 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "72cb131c83b749d486fb6435b22d60b4"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "12bd7325b471b5b83199f1a711fe5cdc"
server_key_iv: "daff92f7e35a7201b3149e80b5eef100"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.300279 2529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:31.300964 2547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.301134 2529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:31.301221 2529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "72cb131c83b749d486fb6435b22d60b4"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "12bd7325b471b5b83199f1a711fe5cdc"
server_key_iv: "daff92f7e35a7201b3149e80b5eef100"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.301275 2529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:31.316059 2529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:31.317911 2529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:31.318012 2529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:31.318176 2529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:31.318403 2529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:31.318431 2529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.318450 2529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:31.318462 2529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.324117 2529 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:46405
I20260810 21:03:31.324180 2671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:46405 every 8 connection(s)
I20260810 21:03:31.324455 2529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:31.329708 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2529
I20260810 21:03:31.329820 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:31.330087 15136 external_mini_cluster.cc:1428] Setting key 3897590f9e5b9f921bb3db8d3bd476f6
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:31.333324 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.325145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:53021 (local address 127.14.200.62:41639)
0810 21:03:31.325190 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.325194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.325405 (+ 211us) server_negotiation.cc:407] Connection header received
0810 21:03:31.325827 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.325828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.325838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.325886 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:31.326739 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.327220 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.328107 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.328299 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.328348 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.330619 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:31.330621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:31.330622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:31.330645 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:31.331952 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.332339 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.332341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.332341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.332399 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.332640 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.332641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.332642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.332752 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:31.332862 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.333130 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.333219 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:31.333491 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.325152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41639 (local address 127.14.200.1:53021)
0810 21:03:31.325298 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.325331 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.325749 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.325938 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.325941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.326487 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:31.326668 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.326672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.327298 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.327299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.328040 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.328041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.328369 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.328370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.328463 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.328989 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:31.328997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:31.330509 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:31.332039 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.332046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.332049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.332265 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.332466 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.332467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.332468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.332575 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.332856 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:31.332863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:31.332992 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.333106 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.333253 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260810 21:03:31.333945 2672 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41639
I20260810 21:03:31.334036 2672 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:31.334225 2672 heartbeater.cc:511] Master 127.14.200.62:41639 requested a full tablet report, sending...
I20260810 21:03:31.334764 2461 ts_manager.cc:194] Registered new tserver with Master: 72cb131c83b749d486fb6435b22d60b4 (127.14.200.1:46405)
I20260810 21:03:31.335742 2461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:53021
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:03:31.355834 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:31.435276 2678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:31.435456 2678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:31.435477 2678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:31.436537 2678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:31.436576 2678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:31.436658 2678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:31.438225 2678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:31.438467 2678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:31.438686 2678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:31.443509 2686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.443518 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.443570 2678 server_base.cc:1034] running on GCE node
W20260810 21:03:31.443519 2684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.443876 2678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:31.444111 2678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:31.445240 2678 hybrid_clock.cc:648] HybridClock initialized: now 1786395811445230 us; error 27 us; skew 500 ppm
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:31.447775 2678 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:03:31.448244 2678 webserver.cc:460] Webserver started at http://127.14.200.2:40637/ using document root <none> and password file <none>
I20260810 21:03:31.448413 2678 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:31.448454 2678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:31.448539 2678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:31.449559 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "ce4cc645ec0b43e484fcff5edf83723d"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "a98883187cece47d9752ef5b898b2a73"
server_key_iv: "b4d1145faf2a3d5d55332a581a970251"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.449946 2678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "ce4cc645ec0b43e484fcff5edf83723d"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "a98883187cece47d9752ef5b898b2a73"
server_key_iv: "b4d1145faf2a3d5d55332a581a970251"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.451241 2678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 21:03:31.451946 2694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.452116 2678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:31.452201 2678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "ce4cc645ec0b43e484fcff5edf83723d"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "a98883187cece47d9752ef5b898b2a73"
server_key_iv: "b4d1145faf2a3d5d55332a581a970251"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.452255 2678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:31.475677 2678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:31.477473 2678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:31.477578 2678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:31.477747 2678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:31.478080 2678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:31.478116 2678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.478170 2678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:31.478204 2678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.483871 2678 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:43503
I20260810 21:03:31.483959 2816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:43503 every 8 connection(s)
I20260810 21:03:31.484301 2678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:31.490490 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2678
I20260810 21:03:31.490618 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:31.490818 15136 external_mini_cluster.cc:1428] Setting key 83a2a93256c6ce57bd78c571a3a10059
I20260810 21:03:31.493805 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.485029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:55559 (local address 127.14.200.62:41639)
0810 21:03:31.485077 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.485079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.485303 (+ 224us) server_negotiation.cc:407] Connection header received
0810 21:03:31.485838 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.485839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.485870 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.485919 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:31.486857 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.487357 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.488311 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.488503 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.488589 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.490820 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:31.490821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:31.490822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:31.490828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:31.492322 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.492828 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.492829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.492829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.492892 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.493191 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.493192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.493192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.493288 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:31.493365 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.493645 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.493721 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:31.493999 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.485011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41639 (local address 127.14.200.2:55559)
0810 21:03:31.485191 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.485224 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.485720 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.485997 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.486001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.486567 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:31.486750 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.486754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.487450 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.487451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.488221 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.488223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.488600 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.488601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.488723 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.489297 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:31.489306 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:31.490711 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:31.492399 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.492407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.492409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.492719 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.492962 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.492963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.492963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.493129 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.493371 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:31.493378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:31.493504 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.493609 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.493760 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260810 21:03:31.494450 2817 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41639
I20260810 21:03:31.494560 2817 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:31.494747 2817 heartbeater.cc:511] Master 127.14.200.62:41639 requested a full tablet report, sending...
I20260810 21:03:31.495111 2461 ts_manager.cc:194] Registered new tserver with Master: ce4cc645ec0b43e484fcff5edf83723d (127.14.200.2:43503)
I20260810 21:03:31.495492 2461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:55559
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:03:31.516698 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:31.596000 2823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:31.596194 2823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:31.596215 2823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:31.597396 2823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:31.597440 2823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:31.597465 2823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:31.598918 2823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:31.599148 2823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:31.599364 2823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:31.604127 2823 server_base.cc:1034] running on GCE node
W20260810 21:03:31.604110 2832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.604094 2828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:31.604316 2830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:31.604557 2823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:31.604847 2823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:31.606025 2823 hybrid_clock.cc:648] HybridClock initialized: now 1786395811606004 us; error 44 us; skew 500 ppm
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:31.608433 2823 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:03:31.608922 2823 webserver.cc:460] Webserver started at http://127.14.200.3:46693/ using document root <none> and password file <none>
I20260810 21:03:31.609107 2823 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:31.609146 2823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:31.609229 2823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:31.610178 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "8556167bd89246d199ec763de8cf9773"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "5add1de175902d6478a29f3b67da9e07"
server_key_iv: "09f7ecd92e6d8572cbc841e09d2bae42"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.610496 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "8556167bd89246d199ec763de8cf9773"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "5add1de175902d6478a29f3b67da9e07"
server_key_iv: "09f7ecd92e6d8572cbc841e09d2bae42"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.611697 2823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:31.612329 2840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.612546 2823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:31.612635 2823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "8556167bd89246d199ec763de8cf9773"
format_stamp: "Formatted at 2026-08-10 21:03:31 on dist-test-slave-1x4g"
server_key: "5add1de175902d6478a29f3b67da9e07"
server_key_iv: "09f7ecd92e6d8572cbc841e09d2bae42"
server_key_version: "encryptionkey@0"
I20260810 21:03:31.612689 2823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:31.627419 2823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:31.629062 2823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:31.629160 2823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:31.629307 2823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:31.629582 2823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:31.629613 2823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.629632 2823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:31.629675 2823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:31.635612 2823 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:32793
I20260810 21:03:31.635658 2963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:32793 every 8 connection(s)
I20260810 21:03:31.635941 2823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:03:31.641477 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 2823
I20260810 21:03:31.641619 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:31.641839 15136 external_mini_cluster.cc:1428] Setting key 70f737cb5fba074e5288b5114df0b42d
Aug 10 21:03:31 dist-test-slave-1x4g krb5kdc[2411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395811, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:31.645407 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.636536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:36877 (local address 127.14.200.62:41639)
0810 21:03:31.636610 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.636613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.636899 (+ 286us) server_negotiation.cc:407] Connection header received
0810 21:03:31.637339 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.637340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.637350 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.637399 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:31.638470 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.638939 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.639810 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.639999 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.640058 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.642535 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:31.642537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:31.642537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:31.642546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:31.643908 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.644415 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.644416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.644416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.644484 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.644730 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.644731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.644731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.644890 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:31.644970 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.645232 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.645314 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 21:03:31.645531 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.636522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41639 (local address 127.14.200.3:36877)
0810 21:03:31.636738 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.636773 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.637259 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.637451 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.637454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.638189 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:31.638376 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.638381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.639017 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.639017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.639752 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.639754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.640089 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.640094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.640211 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.640630 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:31.640638 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:31.642401 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:31.644027 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.644035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.644036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.644311 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.644551 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:31.644552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:31.644552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:31.644657 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:31.644961 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:31.644968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:31.645092 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.645206 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.645340 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":133,"threads_started":1}
I20260810 21:03:31.646000 2964 heartbeater.cc:347] Connected to a master server at 127.14.200.62:41639
I20260810 21:03:31.646109 2964 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:31.646266 2964 heartbeater.cc:511] Master 127.14.200.62:41639 requested a full tablet report, sending...
I20260810 21:03:31.646625 2461 ts_manager.cc:194] Registered new tserver with Master: 8556167bd89246d199ec763de8cf9773 (127.14.200.3:32793)
I20260810 21:03:31.647013 2461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:36877
I20260810 21:03:31.653563 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:31.661109 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.654863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.14.200.62:41639)
0810 21:03:31.654913 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.654915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.655049 (+ 134us) server_negotiation.cc:407] Connection header received
0810 21:03:31.655142 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.655143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.655152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.655228 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:31.655765 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.656297 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.657063 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.657259 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.657331 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.658618 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:31.658619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:31.658619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:31.658626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:31.659856 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.660246 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.660247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.660247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.660308 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:31.660535 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:31.660536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:31.660536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:31.660668 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:31.660782 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.660950 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.661014 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 21:03:31.662611 2461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:31.664153 2461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:31.670231 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43503 (local address 127.0.0.1:48440)
0810 21:03:31.666533 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.666557 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.666621 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.666849 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.666852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.666854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:31.666916 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.666918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668216 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.668217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.669100 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.669103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.669958 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.669959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.670012 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.670021 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.670091 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.670159 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":68,"thread_start_us":64,"threads_started":1}
I20260810 21:03:31.670238 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.14.200.2:43503)
0810 21:03:31.666543 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.666548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.666699 (+ 151us) server_negotiation.cc:407] Connection header received
0810 21:03:31.666748 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.666750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.666762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.666845 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:31.666973 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668121 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.669192 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.669873 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.669948 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.670003 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.670108 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.670151 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260810 21:03:31.670455 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.14.200.3:32793)
0810 21:03:31.666721 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.666725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.666734 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:31.666780 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.666783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.666794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.666879 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:31.667065 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668097 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.669418 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.670051 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.670138 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.670189 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.670326 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.670377 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":76,"threads_started":1}
I20260810 21:03:31.670698 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:32793 (local address 127.0.0.1:60184)
0810 21:03:31.666518 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.666557 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.666621 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.666878 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.666879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.666881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:31.666946 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.666948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668187 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.668188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.669349 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.669352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.670158 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.670158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.670216 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.670218 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.670594 (+ 376us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.670642 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":72,"thread_start_us":75,"threads_started":1}
I20260810 21:03:31.670945 2747 tablet_service.cc:1467] Processing CreateTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 (DEFAULT_TABLE table=test-table [id=cc89132b7a9f48818904b778d20725ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:31.671329 2747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f35d13e2c5462fa1e62bd3e9440a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:31.671389 2891 tablet_service.cc:1467] Processing CreateTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 (DEFAULT_TABLE table=test-table [id=cc89132b7a9f48818904b778d20725ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:31.671758 2891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f35d13e2c5462fa1e62bd3e9440a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:31.672187 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46405 (local address 127.0.0.1:40846)
0810 21:03:31.666597 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.666608 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.666660 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.667200 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.667201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.667203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:31.667249 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.667250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668137 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.668138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.669349 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.669352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.671939 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.671940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.672004 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.672008 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.672082 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.672119 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260810 21:03:31.672403 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.666530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.14.200.1:46405)
0810 21:03:31.667001 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.667005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.667014 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:31.667053 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.667055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.667068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.667140 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:31.667343 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.668044 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.670223 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.671734 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.671854 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.671911 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.672127 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.672178 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":96,"threads_started":1}
I20260810 21:03:31.673216 2600 tablet_service.cc:1467] Processing CreateTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 (DEFAULT_TABLE table=test-table [id=cc89132b7a9f48818904b778d20725ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:31.673532 2600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f35d13e2c5462fa1e62bd3e9440a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:31.674227 2979 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Bootstrap starting.
I20260810 21:03:31.674947 2980 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Bootstrap starting.
I20260810 21:03:31.675103 2979 tablet_bootstrap.cc:654] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:31.675665 2980 tablet_bootstrap.cc:654] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:31.675753 2979 log.cc:824] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:31.676186 2980 log.cc:824] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:31.676507 2984 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Bootstrap starting.
I20260810 21:03:31.676828 2979 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: No bootstrap required, opened a new log
I20260810 21:03:31.676911 2979 ts_tablet_manager.cc:1397] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 21:03:31.677224 2984 tablet_bootstrap.cc:654] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:31.677373 2980 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: No bootstrap required, opened a new log
I20260810 21:03:31.677464 2980 ts_tablet_manager.cc:1397] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 21:03:31.677840 2984 log.cc:824] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:31.678545 2984 tablet_bootstrap.cc:492] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: No bootstrap required, opened a new log
I20260810 21:03:31.678622 2984 ts_tablet_manager.cc:1397] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:31.678771 2979 raft_consensus.cc:348] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.678885 2979 raft_consensus.cc:374] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:31.678905 2979 raft_consensus.cc:729] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4cc645ec0b43e484fcff5edf83723d, State: Initialized, Role: FOLLOWER
I20260810 21:03:31.679018 2979 consensus_queue.cc:260] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.679267 2979 ts_tablet_manager.cc:1428] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:31.679267 2817 heartbeater.cc:503] Master 127.14.200.62:41639 was elected leader, sending a full tablet report...
I20260810 21:03:31.679327 2980 raft_consensus.cc:348] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.679471 2980 raft_consensus.cc:374] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:31.679504 2980 raft_consensus.cc:729] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8556167bd89246d199ec763de8cf9773, State: Initialized, Role: FOLLOWER
I20260810 21:03:31.679605 2980 consensus_queue.cc:260] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.679863 2964 heartbeater.cc:503] Master 127.14.200.62:41639 was elected leader, sending a full tablet report...
I20260810 21:03:31.679850 2980 ts_tablet_manager.cc:1428] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:03:31.680437 2984 raft_consensus.cc:348] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.680552 2984 raft_consensus.cc:374] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:31.680572 2984 raft_consensus.cc:729] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72cb131c83b749d486fb6435b22d60b4, State: Initialized, Role: FOLLOWER
I20260810 21:03:31.680646 2984 consensus_queue.cc:260] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.680881 2984 ts_tablet_manager.cc:1428] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:31.680953 2672 heartbeater.cc:503] Master 127.14.200.62:41639 was elected leader, sending a full tablet report...
W20260810 21:03:31.734899 2819 tablet.cc:2369] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:03:31.825263 2673 tablet.cc:2369] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 21:03:31.886564 2965 tablet.cc:2369] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:03:31.897770 2987 raft_consensus.cc:482] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:31.897889 2987 raft_consensus.cc:504] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.898211 2987 leader_election.cc:288] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8556167bd89246d199ec763de8cf9773 (127.14.200.3:32793), 72cb131c83b749d486fb6435b22d60b4 (127.14.200.1:46405)
I20260810 21:03:31.901008 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.898490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:60643 (local address 127.14.200.3:32793)
0810 21:03:31.898561 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.898564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.898575 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:03:31.898605 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.898606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.898616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.898675 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:31.898829 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.899301 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900210 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.900607 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900695 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.900704 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.900905 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.900941 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 21:03:31.901008 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.898387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:32793 (local address 127.14.200.2:60643)
0810 21:03:31.898455 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.898468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.898530 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.898680 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.898681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.898682 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:31.898750 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.898752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.899383 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.899384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.900117 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.900119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.900692 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.900832 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.900836 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.900896 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.900941 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 21:03:31.901160 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.898498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46405 (local address 127.14.200.2:54565)
0810 21:03:31.898659 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.898673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:31.898730 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:31.898876 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:31.898877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:31.898879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:31.898961 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.898963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.899562 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.899563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.900295 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:31.900297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.900896 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:31.900971 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.900974 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:31.901043 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 21:03:31.901080 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260810 21:03:31.901329 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.898574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:54565 (local address 127.14.200.1:46405)
0810 21:03:31.898617 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.898620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.898728 (+ 108us) server_negotiation.cc:407] Connection header received
0810 21:03:31.898789 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.898791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.898802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.898872 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:31.899035 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.899477 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900386 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.900753 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.900880 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.900889 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.901049 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.901238 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 21:03:31.901435 2914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f35d13e2c5462fa1e62bd3e9440a06" candidate_uuid: "ce4cc645ec0b43e484fcff5edf83723d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8556167bd89246d199ec763de8cf9773" is_pre_election: true
I20260810 21:03:31.901575 2914 raft_consensus.cc:2454] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce4cc645ec0b43e484fcff5edf83723d in term 0.
I20260810 21:03:31.901798 2696 leader_election.cc:302] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8556167bd89246d199ec763de8cf9773, ce4cc645ec0b43e484fcff5edf83723d; no voters:
I20260810 21:03:31.901947 2987 raft_consensus.cc:2781] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:31.901916 2622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f35d13e2c5462fa1e62bd3e9440a06" candidate_uuid: "ce4cc645ec0b43e484fcff5edf83723d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72cb131c83b749d486fb6435b22d60b4" is_pre_election: true
I20260810 21:03:31.902011 2987 raft_consensus.cc:482] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:31.902041 2987 raft_consensus.cc:3037] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:31.902076 2622 raft_consensus.cc:2454] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce4cc645ec0b43e484fcff5edf83723d in term 0.
I20260810 21:03:31.902805 2987 raft_consensus.cc:504] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.902942 2987 leader_election.cc:288] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [CANDIDATE]: Term 1 election: Requested vote from peers 8556167bd89246d199ec763de8cf9773 (127.14.200.3:32793), 72cb131c83b749d486fb6435b22d60b4 (127.14.200.1:46405)
I20260810 21:03:31.903131 2914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f35d13e2c5462fa1e62bd3e9440a06" candidate_uuid: "ce4cc645ec0b43e484fcff5edf83723d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8556167bd89246d199ec763de8cf9773"
I20260810 21:03:31.903151 2622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f35d13e2c5462fa1e62bd3e9440a06" candidate_uuid: "ce4cc645ec0b43e484fcff5edf83723d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72cb131c83b749d486fb6435b22d60b4"
I20260810 21:03:31.903213 2914 raft_consensus.cc:3037] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:31.903247 2622 raft_consensus.cc:3037] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:31.903963 2622 raft_consensus.cc:2454] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4cc645ec0b43e484fcff5edf83723d in term 1.
I20260810 21:03:31.904016 2914 raft_consensus.cc:2454] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce4cc645ec0b43e484fcff5edf83723d in term 1.
I20260810 21:03:31.904251 2696 leader_election.cc:302] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8556167bd89246d199ec763de8cf9773, ce4cc645ec0b43e484fcff5edf83723d; no voters:
I20260810 21:03:31.904389 2987 raft_consensus.cc:2781] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:31.904580 2987 raft_consensus.cc:686] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 1 LEADER]: Becoming Leader. State: Replica: ce4cc645ec0b43e484fcff5edf83723d, State: Running, Role: LEADER
I20260810 21:03:31.904685 2987 consensus_queue.cc:237] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } }
I20260810 21:03:31.905499 2460 catalog_manager.cc:5495] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce4cc645ec0b43e484fcff5edf83723d (127.14.200.2). New cstate: current_term: 1 leader_uuid: "ce4cc645ec0b43e484fcff5edf83723d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce4cc645ec0b43e484fcff5edf83723d" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43503 } health_report { overall_health: HEALTHY } } }
I20260810 21:03:31.951153 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.948976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.14.200.2:43503)
0810 21:03:31.949041 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:31.949043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:31.949064 (+ 21us) server_negotiation.cc:407] Connection header received
0810 21:03:31.949133 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:31.949134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:31.949155 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:31.949209 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:31.949324 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.949786 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.950500 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:31.950680 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:31.950728 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:31.950856 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:31.950947 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:31.951017 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:31.951066 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 21:03:31.951076 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 21:03:31.952982 2622 raft_consensus.cc:1261] T 99f35d13e2c5462fa1e62bd3e9440a06 P 72cb131c83b749d486fb6435b22d60b4 [term 1 FOLLOWER]: Refusing update from remote peer ce4cc645ec0b43e484fcff5edf83723d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:31.952982 2914 raft_consensus.cc:1261] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 1 FOLLOWER]: Refusing update from remote peer ce4cc645ec0b43e484fcff5edf83723d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:31.953378 2987 consensus_queue.cc:1035] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8556167bd89246d199ec763de8cf9773" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 32793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:31.953523 2991 consensus_queue.cc:1035] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [LEADER]: Connected to new peer: Peer: permanent_uuid: "72cb131c83b749d486fb6435b22d60b4" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 46405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:31.956215 2995 mvcc.cc:205] Tried to move back new op lower bound from 7317077245756297216 to 7317077245562368000. Current Snapshot: MvccSnapshot[applied={T|T < 7317077245756297216}]
I20260810 21:03:31.956861 2996 mvcc.cc:205] Tried to move back new op lower bound from 7317077245756297216 to 7317077245562368000. Current Snapshot: MvccSnapshot[applied={T|T < 7317077245756297216}]
I20260810 21:03:31.958705 2460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:31.958807 2460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:31.959600 2460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93947dc3202340c8b267be49789c823b: Sending DeleteTablet for 3 replicas of tablet 99f35d13e2c5462fa1e62bd3e9440a06
I20260810 21:03:31.959923 2747 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:31 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:48440
I20260810 21:03:31.959946 2891 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:31 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:60184
I20260810 21:03:31.959990 2600 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f35d13e2c5462fa1e62bd3e9440a06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:31 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:40846
I20260810 21:03:31.960168 3005 tablet_replica.cc:330] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: stopping tablet replica
I20260810 21:03:31.960276 3003 tablet_replica.cc:330] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: stopping tablet replica
I20260810 21:03:31.960389 3005 raft_consensus.cc:2229] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:31.960471 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2529
I20260810 21:03:31.960417 3003 raft_consensus.cc:2229] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:31.963965 3003 raft_consensus.cc:2258] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:31.964394 3003 ts_tablet_manager.cc:1905] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:31.966220 3003 ts_tablet_manager.cc:1918] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:31.966269 3003 log.cc:1196] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/99f35d13e2c5462fa1e62bd3e9440a06
I20260810 21:03:31.966459 3003 ts_tablet_manager.cc:1939] T 99f35d13e2c5462fa1e62bd3e9440a06 P ce4cc645ec0b43e484fcff5edf83723d: Deleting consensus metadata
I20260810 21:03:31.966811 2447 catalog_manager.cc:4841] TS ce4cc645ec0b43e484fcff5edf83723d (127.14.200.2:43503): tablet 99f35d13e2c5462fa1e62bd3e9440a06 (table test-table [id=cc89132b7a9f48818904b778d20725ca]) successfully deleted
I20260810 21:03:31.969640 3005 raft_consensus.cc:2258] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:31.970124 3005 ts_tablet_manager.cc:1905] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:31.971706 3005 ts_tablet_manager.cc:1918] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:31.971781 3005 log.cc:1196] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/99f35d13e2c5462fa1e62bd3e9440a06
I20260810 21:03:31.972041 3005 ts_tablet_manager.cc:1939] T 99f35d13e2c5462fa1e62bd3e9440a06 P 8556167bd89246d199ec763de8cf9773: Deleting consensus metadata
I20260810 21:03:31.972442 2445 catalog_manager.cc:4841] TS 8556167bd89246d199ec763de8cf9773 (127.14.200.3:32793): tablet 99f35d13e2c5462fa1e62bd3e9440a06 (table test-table [id=cc89132b7a9f48818904b778d20725ca]) successfully deleted
W20260810 21:03:31.974359 2445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:46405 (error 108)
I20260810 21:03:31.974593 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2678
I20260810 21:03:31.975602 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:31.974725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:46405 (local address 127.0.0.1:40852)
0810 21:03:31.975536 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:31.975574 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":789}
W20260810 21:03:31.981267 2445 catalog_manager.cc:4584] TS 72cb131c83b749d486fb6435b22d60b4 (127.14.200.1:46405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99f35d13e2c5462fa1e62bd3e9440a06: Network error: Client connection negotiation failed: client connection to 127.14.200.1:46405: connect: Connection refused (error 111)
I20260810 21:03:31.984347 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2823
I20260810 21:03:31.990512 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 2427
2026-08-10T21:03:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2181 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T21:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:32Z Disabled control of system clock
I20260810 21:03:32.020768 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35659
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:39819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:35659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:32.099474 3012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:32.099679 3012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:32.099699 3012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:32.100754 3012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:32.100801 3012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:32.100816 3012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:32.100827 3012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:32.100878 3012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:32.102151 3012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:35659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:32.102321 3012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:32.102532 3012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:32.107254 3019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.107242 3021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:32.107287 3012 server_base.cc:1034] running on GCE node
W20260810 21:03:32.107276 3018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:32.107731 3012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:32.108009 3012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:32.109181 3012 hybrid_clock.cc:648] HybridClock initialized: now 1786395812109161 us; error 54 us; skew 500 ppm
I20260810 21:03:32.114841 3012 webserver.cc:460] Webserver started at http://127.14.200.62:35387/ using document root <none> and password file <none>
I20260810 21:03:32.115080 3012 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:32.115159 3012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:32.115302 3012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:32.116276 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "cb8a64a52f8943d8ae3a3a5848de28d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "d0f51010d6be3898585c24fdafdbe5e8"
server_key_iv: "98bc1fdafc19a8fd7d48f29a395b30f3"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.116657 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "cb8a64a52f8943d8ae3a3a5848de28d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "d0f51010d6be3898585c24fdafdbe5e8"
server_key_iv: "98bc1fdafc19a8fd7d48f29a395b30f3"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.117976 3012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:03:32.118673 3027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.118834 3012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:32.118932 3012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "cb8a64a52f8943d8ae3a3a5848de28d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "d0f51010d6be3898585c24fdafdbe5e8"
server_key_iv: "98bc1fdafc19a8fd7d48f29a395b30f3"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.119009 3012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:32.130100 3012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:32.130363 3012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:32.130509 3012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:32.134433 3012 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:35659
I20260810 21:03:32.134471 3085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:35659 every 8 connection(s)
I20260810 21:03:32.134826 3012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:32.135087 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3012
I20260810 21:03:32.135198 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:32.135394 15136 external_mini_cluster.cc:1428] Setting key fadf3a3afc9412b272760ed785f1cfc2
I20260810 21:03:32.135905 3086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:32.138492 3086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: Bootstrap starting.
I20260810 21:03:32.139102 3086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:32.139381 3086 log.cc:824] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:32.140141 3086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: No bootstrap required, opened a new log
I20260810 21:03:32.140971 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.135920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.14.200.62:35659)
0810 21:03:32.136150 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.136162 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.136185 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:03:32.136570 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.136576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.136658 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.136823 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:32.137062 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.137925 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.138780 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.139202 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.139334 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.140329 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:32.140333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:32.140333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:32.140482 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:32.140512 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:32.140522 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:32.140595 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.140670 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.140745 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260810 21:03:32.142282 3086 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } }
I20260810 21:03:32.142424 3086 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:32.142470 3086 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb8a64a52f8943d8ae3a3a5848de28d6, State: Initialized, Role: FOLLOWER
I20260810 21:03:32.142599 3086 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } }
I20260810 21:03:32.142692 3086 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:32.142735 3086 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:32.142782 3086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:32.143265 3086 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } }
I20260810 21:03:32.143381 3086 leader_election.cc:302] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb8a64a52f8943d8ae3a3a5848de28d6; no voters:
I20260810 21:03:32.143573 3086 leader_election.cc:288] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:32.143613 3091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:32.143769 3091 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [term 1 LEADER]: Becoming Leader. State: Replica: cb8a64a52f8943d8ae3a3a5848de28d6, State: Running, Role: LEADER
I20260810 21:03:32.143833 3086 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:32.143848 3091 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } }
I20260810 21:03:32.144270 3092 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } } }
I20260810 21:03:32.144284 3094 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb8a64a52f8943d8ae3a3a5848de28d6. Latest consensus state: current_term: 1 leader_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8a64a52f8943d8ae3a3a5848de28d6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35659 } } }
I20260810 21:03:32.144477 3094 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:32.144461 3092 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:32.144837 3101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:32.145290 3101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:32.146796 3101 catalog_manager.cc:1347] Generated new cluster ID: 9983fac8332c4855b3d065a47212f208
I20260810 21:03:32.146852 3101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:32.153219 3101 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:32.153667 3101 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:32.161482 3101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: Generated new TSK 0
I20260810 21:03:32.161659 3101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:32.169430 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--builtin_ntp_servers=127.14.200.20:39819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:32.251286 3111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:32.251507 3111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:32.251540 3111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:32.252642 3111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:32.252696 3111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:32.252732 3111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:32.254050 3111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:32.254285 3111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:32.254568 3111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:32.259374 3117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.259411 3116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.259364 3119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:32.259709 3111 server_base.cc:1034] running on GCE node
I20260810 21:03:32.259948 3111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:32.260208 3111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:32.261356 3111 hybrid_clock.cc:648] HybridClock initialized: now 1786395812261342 us; error 31 us; skew 500 ppm
I20260810 21:03:32.262615 3111 webserver.cc:460] Webserver started at http://127.14.200.1:35551/ using document root <none> and password file <none>
I20260810 21:03:32.262866 3111 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:32.262936 3111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:32.263095 3111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:32.264101 3111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "e58ac5ad71db458285011121201bf9d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "0c3109998c8b28a67082147e3ae21ad8"
server_key_iv: "3157b0f1f2deafc6972d2290e95e571d"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.264453 3111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "e58ac5ad71db458285011121201bf9d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "0c3109998c8b28a67082147e3ae21ad8"
server_key_iv: "3157b0f1f2deafc6972d2290e95e571d"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.265944 3111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:32.266995 3125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.267210 3111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:03:32.267266 3111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "e58ac5ad71db458285011121201bf9d6"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "0c3109998c8b28a67082147e3ae21ad8"
server_key_iv: "3157b0f1f2deafc6972d2290e95e571d"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.267347 3111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:32.276461 3111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:32.276749 3111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:32.276949 3111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:32.277168 3111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:32.277479 3111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:32.277513 3111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.277556 3111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:32.277582 3111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.283421 3111 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:34783
I20260810 21:03:32.283468 3249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:34783 every 8 connection(s)
I20260810 21:03:32.283746 3111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:32.285130 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3111
I20260810 21:03:32.285247 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:32.285434 15136 external_mini_cluster.cc:1428] Setting key 261b23b3a6a1028c5aa83e5410c830f2
I20260810 21:03:32.286412 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--builtin_ntp_servers=127.14.200.20:39819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 21:03:32.289026 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.284368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:46527 (local address 127.14.200.62:35659)
0810 21:03:32.284414 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.284417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.284691 (+ 274us) server_negotiation.cc:407] Connection header received
0810 21:03:32.285185 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.285187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.285202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.285262 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:32.286027 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.286631 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.287580 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.287831 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.287906 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.288703 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:32.288705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:32.288705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:32.288710 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:32.288727 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:32.288733 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:32.288837 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.288944 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.288975 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:32.289250 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.284380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35659 (local address 127.14.200.1:46527)
0810 21:03:32.284582 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.284611 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.285085 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.285334 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.285338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.285351 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:32.285933 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.285938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.286733 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.286734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.287470 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.287472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.287947 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.287948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.288084 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.288573 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:32.288573 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:32.288598 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:32.288619 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:32.288861 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:32.288868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:32.288870 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.288941 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.289057 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260810 21:03:32.289770 3250 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35659
I20260810 21:03:32.289862 3250 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:32.290057 3250 heartbeater.cc:511] Master 127.14.200.62:35659 requested a full tablet report, sending...
I20260810 21:03:32.290616 3044 ts_manager.cc:194] Registered new tserver with Master: e58ac5ad71db458285011121201bf9d6 (127.14.200.1:34783)
I20260810 21:03:32.291567 3044 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.1:46527
W20260810 21:03:32.366762 3253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:32.366927 3253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:32.366953 3253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:32.368251 3253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:32.368290 3253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:32.368314 3253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:32.369592 3253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:32.369830 3253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:32.370117 3253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:32.375026 3260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.375073 3259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.375090 3262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:32.375478 3253 server_base.cc:1034] running on GCE node
I20260810 21:03:32.375731 3253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:32.375957 3253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:32.377110 3253 hybrid_clock.cc:648] HybridClock initialized: now 1786395812377084 us; error 36 us; skew 500 ppm
I20260810 21:03:32.378404 3253 webserver.cc:460] Webserver started at http://127.14.200.2:35205/ using document root <none> and password file <none>
I20260810 21:03:32.378660 3253 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:32.378731 3253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:32.378867 3253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:32.379827 3253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "70da0c0dd4114a9f9460fdf291f9ae62"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "f7d938fab3d46c62e970163179c47132"
server_key_iv: "c2db2a715ad491eeda0b8211ee470e44"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.380213 3253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "70da0c0dd4114a9f9460fdf291f9ae62"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "f7d938fab3d46c62e970163179c47132"
server_key_iv: "c2db2a715ad491eeda0b8211ee470e44"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.381604 3253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:32.382336 3268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.382488 3253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:32.382596 3253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "70da0c0dd4114a9f9460fdf291f9ae62"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "f7d938fab3d46c62e970163179c47132"
server_key_iv: "c2db2a715ad491eeda0b8211ee470e44"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.382666 3253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:32.399591 3253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:32.399856 3253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:32.400002 3253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:32.400231 3253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:32.400539 3253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:32.400594 3253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.400627 3253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:32.400655 3253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.406479 3253 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:43963
I20260810 21:03:32.406529 3390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:43963 every 8 connection(s)
I20260810 21:03:32.406847 3253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:03:32.412165 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.407592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:42289 (local address 127.14.200.62:35659)
0810 21:03:32.407686 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.407689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.407871 (+ 182us) server_negotiation.cc:407] Connection header received
0810 21:03:32.408396 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.408398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.408413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.408483 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:32.409277 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.409798 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.410802 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.411018 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.411102 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.411858 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:32.411860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:32.411860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:32.411866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:32.411885 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:32.411890 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:32.411950 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.412053 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.412082 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 21:03:32.412377 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.407512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35659 (local address 127.14.200.2:42289)
0810 21:03:32.407748 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.407780 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.408269 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.408567 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.408570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.408577 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:32.409166 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.409172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.409913 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.409915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.410686 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.410689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.411111 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.411113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.411259 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.411699 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:32.411700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:32.411725 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:32.411760 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:32.411954 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:32.411960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:32.411963 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.412055 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.412154 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260810 21:03:32.412398 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3253
I20260810 21:03:32.412504 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:32.412735 15136 external_mini_cluster.cc:1428] Setting key ddf312d099fe4648c35a3c1b53ee5b18
I20260810 21:03:32.412974 3391 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35659
I20260810 21:03:32.413072 3391 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:32.413272 3391 heartbeater.cc:511] Master 127.14.200.62:35659 requested a full tablet report, sending...
I20260810 21:03:32.413637 3044 ts_manager.cc:194] Registered new tserver with Master: 70da0c0dd4114a9f9460fdf291f9ae62 (127.14.200.2:43963)
I20260810 21:03:32.413782 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--builtin_ntp_servers=127.14.200.20:39819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 21:03:32.414091 3044 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.2:42289
W20260810 21:03:32.490834 3394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:32.491050 3394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:32.491086 3394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:32.492172 3394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:32.492231 3394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:32.492273 3394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:32.493637 3394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:35659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:32.493889 3394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:32.494168 3394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:32.499115 3400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.499161 3399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:32.499150 3402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:32.499193 3394 server_base.cc:1034] running on GCE node
I20260810 21:03:32.499501 3394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:32.499743 3394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:32.500953 3394 hybrid_clock.cc:648] HybridClock initialized: now 1786395812500930 us; error 37 us; skew 500 ppm
I20260810 21:03:32.502211 3394 webserver.cc:460] Webserver started at http://127.14.200.3:39611/ using document root <none> and password file <none>
I20260810 21:03:32.502475 3394 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:32.502532 3394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:32.502713 3394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:32.503720 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "0e60debca709465c95416a933591bddf"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "906cca4c36106d1106e9355a8797875a"
server_key_iv: "3ff6c13211f922cd355737eb0957d982"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.504092 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "0e60debca709465c95416a933591bddf"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "906cca4c36106d1106e9355a8797875a"
server_key_iv: "3ff6c13211f922cd355737eb0957d982"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.505496 3394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:32.506198 3408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.506363 3394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:32.506477 3394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "0e60debca709465c95416a933591bddf"
format_stamp: "Formatted at 2026-08-10 21:03:32 on dist-test-slave-1x4g"
server_key: "906cca4c36106d1106e9355a8797875a"
server_key_iv: "3ff6c13211f922cd355737eb0957d982"
server_key_version: "encryptionkey@0"
I20260810 21:03:32.506558 3394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:32.519309 3394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:32.519586 3394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:32.519730 3394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:32.519937 3394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:32.520272 3394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:32.520328 3394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.520368 3394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:32.520398 3394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:32.526472 3394 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:33717
I20260810 21:03:32.526520 3532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:33717 every 8 connection(s)
I20260810 21:03:32.526834 3394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:03:32.529181 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3394
I20260810 21:03:32.529309 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:32.529510 15136 external_mini_cluster.cc:1428] Setting key ba46e0661c3a473b2cc31f70adbdad70
I20260810 21:03:32.532202 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.527539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38251 (local address 127.14.200.62:35659)
0810 21:03:32.527607 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.527609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.527836 (+ 227us) server_negotiation.cc:407] Connection header received
0810 21:03:32.528366 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.528367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.528380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.528431 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:32.529197 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.529917 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.531061 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.531245 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.531298 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.531999 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:32.532000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:32.532001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:32.532005 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:32.532019 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:32.532023 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:32.532065 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.532126 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.532146 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 21:03:32.532425 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.527516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:35659 (local address 127.14.200.3:38251)
0810 21:03:32.527720 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.527753 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.528251 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.528495 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.528499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.528508 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:32.529103 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.529108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.530012 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.530013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.530982 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.530985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.531317 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.531318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.531421 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.531866 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:32.531866 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:32.531893 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:32.531915 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:32.532065 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:32.532067 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:32.532069 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.532114 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.532187 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260810 21:03:32.532888 3533 heartbeater.cc:347] Connected to a master server at 127.14.200.62:35659
I20260810 21:03:32.532987 3533 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:32.533165 3533 heartbeater.cc:511] Master 127.14.200.62:35659 requested a full tablet report, sending...
I20260810 21:03:32.533536 3044 ts_manager.cc:194] Registered new tserver with Master: 0e60debca709465c95416a933591bddf (127.14.200.3:33717)
I20260810 21:03:32.533898 3044 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.3:38251
I20260810 21:03:32.541137 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:32.545394 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.542496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.14.200.62:35659)
0810 21:03:32.542554 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.542556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.542697 (+ 141us) server_negotiation.cc:407] Connection header received
0810 21:03:32.542732 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.542733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.542741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.542806 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:32.542970 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.543485 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.544200 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.544373 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.544453 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.545208 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:32.545209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:32.545209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:32.545213 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:32.545223 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:32.545227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:32.545291 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.545335 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.545344 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 21:03:32.546859 3044 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:32.548431 3044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:32.554621 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34783 (local address 127.0.0.1:54338)
0810 21:03:32.551124 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.551134 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.551178 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.551434 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.551435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.551437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:32.551482 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.551484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552336 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.552338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.553614 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.553617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554335 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.554412 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554426 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.554499 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.554548 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260810 21:03:32.554634 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.14.200.1:34783)
0810 21:03:32.550944 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.550948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.551233 (+ 285us) server_negotiation.cc:407] Connection header received
0810 21:03:32.551280 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.551282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.551296 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.551364 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:32.551565 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552245 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.553716 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554232 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554320 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554415 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.554514 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.554554 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
I20260810 21:03:32.554816 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:33717 (local address 127.0.0.1:42372)
0810 21:03:32.550974 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.550985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.551043 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.551281 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.551282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.551283 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:32.551320 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.551321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552581 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.552582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.553592 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.553594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554573 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.554636 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554640 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.554690 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.554735 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260810 21:03:32.555059 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.14.200.3:33717)
0810 21:03:32.550833 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.550839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.551060 (+ 221us) server_negotiation.cc:407] Connection header received
0810 21:03:32.551111 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.551113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.551146 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.551224 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:32.551394 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552488 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.553677 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554362 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554435 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554534 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.554942 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.554984 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260810 21:03:32.555075 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36948 (local address 127.14.200.2:43963)
0810 21:03:32.550927 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.550931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.550943 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:03:32.551042 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.551044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.551060 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.551146 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:32.551348 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552476 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.553743 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554454 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554514 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554589 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.554854 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.554985 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260810 21:03:32.554814 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.550476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43963 (local address 127.0.0.1:36948)
0810 21:03:32.550830 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.550870 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.550940 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.551164 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.551168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.551171 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:32.551256 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.551258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.552572 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.552574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.553676 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.553678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.554618 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.554619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.554675 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.554679 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.554724 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.554752 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":20,"thread_start_us":69,"threads_started":1}
I20260810 21:03:32.555671 3176 tablet_service.cc:1467] Processing CreateTablet for tablet d08946839aa04f8d8017b5e8561af5f6 (DEFAULT_TABLE table=test-table [id=af348f0688f8402aa099a081994b8e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:32.555786 3319 tablet_service.cc:1467] Processing CreateTablet for tablet d08946839aa04f8d8017b5e8561af5f6 (DEFAULT_TABLE table=test-table [id=af348f0688f8402aa099a081994b8e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:32.556082 3319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d08946839aa04f8d8017b5e8561af5f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:32.556082 3176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d08946839aa04f8d8017b5e8561af5f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:32.557116 3461 tablet_service.cc:1467] Processing CreateTablet for tablet d08946839aa04f8d8017b5e8561af5f6 (DEFAULT_TABLE table=test-table [id=af348f0688f8402aa099a081994b8e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:32.557433 3461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d08946839aa04f8d8017b5e8561af5f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:32.559528 3549 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Bootstrap starting.
I20260810 21:03:32.559770 3548 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: Bootstrap starting.
I20260810 21:03:32.560300 3551 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Bootstrap starting.
I20260810 21:03:32.560491 3548 tablet_bootstrap.cc:654] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:32.560875 3548 log.cc:824] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:32.560966 3551 tablet_bootstrap.cc:654] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:32.561298 3551 log.cc:824] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:32.561455 3549 tablet_bootstrap.cc:654] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:32.561717 3548 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: No bootstrap required, opened a new log
I20260810 21:03:32.561765 3549 log.cc:824] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:32.561920 3548 ts_tablet_manager.cc:1397] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 21:03:32.562036 3551 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: No bootstrap required, opened a new log
I20260810 21:03:32.562108 3551 ts_tablet_manager.cc:1397] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:03:32.562759 3549 tablet_bootstrap.cc:492] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: No bootstrap required, opened a new log
I20260810 21:03:32.562846 3549 ts_tablet_manager.cc:1397] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 21:03:32.564069 3551 raft_consensus.cc:348] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.564217 3551 raft_consensus.cc:374] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:32.564261 3551 raft_consensus.cc:729] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e60debca709465c95416a933591bddf, State: Initialized, Role: FOLLOWER
I20260810 21:03:32.564370 3551 consensus_queue.cc:260] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.564646 3551 ts_tablet_manager.cc:1428] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:03:32.564590 3548 raft_consensus.cc:348] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.564751 3548 raft_consensus.cc:374] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:32.564800 3533 heartbeater.cc:503] Master 127.14.200.62:35659 was elected leader, sending a full tablet report...
I20260810 21:03:32.564817 3548 raft_consensus.cc:729] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58ac5ad71db458285011121201bf9d6, State: Initialized, Role: FOLLOWER
I20260810 21:03:32.564927 3548 consensus_queue.cc:260] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.565183 3548 ts_tablet_manager.cc:1428] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 21:03:32.565307 3250 heartbeater.cc:503] Master 127.14.200.62:35659 was elected leader, sending a full tablet report...
I20260810 21:03:32.565467 3549 raft_consensus.cc:348] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.565625 3549 raft_consensus.cc:374] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:32.565665 3549 raft_consensus.cc:729] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70da0c0dd4114a9f9460fdf291f9ae62, State: Initialized, Role: FOLLOWER
I20260810 21:03:32.565783 3549 consensus_queue.cc:260] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.566258 3549 ts_tablet_manager.cc:1428] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 21:03:32.566413 3391 heartbeater.cc:503] Master 127.14.200.62:35659 was elected leader, sending a full tablet report...
W20260810 21:03:32.657372 3392 tablet.cc:2369] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:03:32.706871 3555 raft_consensus.cc:482] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:32.707019 3555 raft_consensus.cc:504] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.707414 3555 leader_election.cc:288] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70da0c0dd4114a9f9460fdf291f9ae62 (127.14.200.2:43963), e58ac5ad71db458285011121201bf9d6 (127.14.200.1:34783)
I20260810 21:03:32.710409 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.707696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:47967 (local address 127.14.200.1:34783)
0810 21:03:32.707744 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.707746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.707875 (+ 129us) server_negotiation.cc:407] Connection header received
0810 21:03:32.707948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.707949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.707964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.708061 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:32.708215 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.708679 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.709639 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.710031 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.710118 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.710127 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.710297 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.710333 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 21:03:32.710428 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.707653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:58549 (local address 127.14.200.2:43963)
0810 21:03:32.707710 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.707712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.707723 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:03:32.707810 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.707811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.707821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.707899 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:32.708123 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.708590 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.709610 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.710056 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.710146 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.710154 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.710321 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.710359 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:03:32.710433 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.707594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:43963 (local address 127.14.200.3:58549)
0810 21:03:32.707656 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.707667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.707713 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.707908 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.707910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.707911 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:32.707986 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.707988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.708681 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.708682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.709510 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.709513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.710147 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.710149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.710236 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.710239 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.710311 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.710352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 21:03:32.710439 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.707618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34783 (local address 127.14.200.3:47967)
0810 21:03:32.707802 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.707816 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:32.707883 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:32.708064 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:32.708065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:32.708067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:32.708151 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.708152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.708792 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.708793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.709555 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:32.709557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.710116 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.710117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:32.710214 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.710217 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:32.710279 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 21:03:32.710318 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260810 21:03:32.710790 3339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d08946839aa04f8d8017b5e8561af5f6" candidate_uuid: "0e60debca709465c95416a933591bddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" is_pre_election: true
I20260810 21:03:32.710956 3339 raft_consensus.cc:2454] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e60debca709465c95416a933591bddf in term 0.
I20260810 21:03:32.710912 3198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d08946839aa04f8d8017b5e8561af5f6" candidate_uuid: "0e60debca709465c95416a933591bddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58ac5ad71db458285011121201bf9d6" is_pre_election: true
I20260810 21:03:32.711054 3198 raft_consensus.cc:2454] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e60debca709465c95416a933591bddf in term 0.
I20260810 21:03:32.711251 3411 leader_election.cc:302] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e60debca709465c95416a933591bddf, 70da0c0dd4114a9f9460fdf291f9ae62; no voters:
I20260810 21:03:32.711370 3555 raft_consensus.cc:2781] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:32.711427 3555 raft_consensus.cc:482] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:32.711472 3555 raft_consensus.cc:3037] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:32.712167 3555 raft_consensus.cc:504] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.712306 3555 leader_election.cc:288] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [CANDIDATE]: Term 1 election: Requested vote from peers 70da0c0dd4114a9f9460fdf291f9ae62 (127.14.200.2:43963), e58ac5ad71db458285011121201bf9d6 (127.14.200.1:34783)
I20260810 21:03:32.712499 3198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d08946839aa04f8d8017b5e8561af5f6" candidate_uuid: "0e60debca709465c95416a933591bddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58ac5ad71db458285011121201bf9d6"
I20260810 21:03:32.712584 3198 raft_consensus.cc:3037] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:32.712882 3339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d08946839aa04f8d8017b5e8561af5f6" candidate_uuid: "0e60debca709465c95416a933591bddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70da0c0dd4114a9f9460fdf291f9ae62"
I20260810 21:03:32.712983 3339 raft_consensus.cc:3037] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:32.713430 3198 raft_consensus.cc:2454] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e60debca709465c95416a933591bddf in term 1.
I20260810 21:03:32.713652 3410 leader_election.cc:302] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e60debca709465c95416a933591bddf, e58ac5ad71db458285011121201bf9d6; no voters:
I20260810 21:03:32.713755 3555 raft_consensus.cc:2781] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:32.713881 3339 raft_consensus.cc:2454] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e60debca709465c95416a933591bddf in term 1.
I20260810 21:03:32.713932 3555 raft_consensus.cc:686] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 1 LEADER]: Becoming Leader. State: Replica: 0e60debca709465c95416a933591bddf, State: Running, Role: LEADER
I20260810 21:03:32.714015 3555 consensus_queue.cc:237] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } }
I20260810 21:03:32.714748 3044 catalog_manager.cc:5495] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e60debca709465c95416a933591bddf (127.14.200.3). New cstate: current_term: 1 leader_uuid: "0e60debca709465c95416a933591bddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e60debca709465c95416a933591bddf" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 33717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 } health_report { overall_health: UNKNOWN } } }
I20260810 21:03:32.735909 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.733423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.14.200.3:33717)
0810 21:03:32.733491 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:32.733493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:32.733502 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:32.733541 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:32.733542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:32.733567 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:32.733663 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:32.733806 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.734274 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.735131 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:32.735321 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:32.735408 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:32.735547 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:32.735637 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:32.735746 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:32.735797 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 21:03:32.735806 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 21:03:32.737707 3198 raft_consensus.cc:1261] T d08946839aa04f8d8017b5e8561af5f6 P e58ac5ad71db458285011121201bf9d6 [term 1 FOLLOWER]: Refusing update from remote peer 0e60debca709465c95416a933591bddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:32.737720 3339 raft_consensus.cc:1261] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 1 FOLLOWER]: Refusing update from remote peer 0e60debca709465c95416a933591bddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:32.738099 3555 consensus_queue.cc:1035] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58ac5ad71db458285011121201bf9d6" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:32.738247 3560 consensus_queue.cc:1035] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "70da0c0dd4114a9f9460fdf291f9ae62" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:32.741004 3564 mvcc.cc:205] Tried to move back new op lower bound from 7317077248970862592 to 7317077248877133824. Current Snapshot: MvccSnapshot[applied={T|T < 7317077248970862592}]
I20260810 21:03:32.743177 3044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:32.743284 3044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:32.754709 3044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb8a64a52f8943d8ae3a3a5848de28d6: Sending DeleteTablet for 3 replicas of tablet d08946839aa04f8d8017b5e8561af5f6
I20260810 21:03:32.755095 3176 tablet_service.cc:1514] Processing DeleteTablet for tablet d08946839aa04f8d8017b5e8561af5f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:32 UTC) from {username='slave'} at 127.0.0.1:54338
I20260810 21:03:32.755121 3461 tablet_service.cc:1514] Processing DeleteTablet for tablet d08946839aa04f8d8017b5e8561af5f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:32 UTC) from {username='slave'} at 127.0.0.1:42372
I20260810 21:03:32.755434 3574 tablet_replica.cc:330] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: stopping tablet replica
I20260810 21:03:32.755461 3319 tablet_service.cc:1514] Processing DeleteTablet for tablet d08946839aa04f8d8017b5e8561af5f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:32 UTC) from {username='slave'} at 127.0.0.1:36948
I20260810 21:03:32.755510 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3111
I20260810 21:03:32.755697 3574 raft_consensus.cc:2229] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:32.755860 3574 raft_consensus.cc:2258] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:32.759258 3574 ts_tablet_manager.cc:1905] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:32.761152 3576 tablet_replica.cc:330] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: stopping tablet replica
I20260810 21:03:32.761279 3576 raft_consensus.cc:2229] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:32.761325 3574 ts_tablet_manager.cc:1918] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:32.761412 3576 raft_consensus.cc:2258] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:32.761518 3574 log.cc:1196] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/d08946839aa04f8d8017b5e8561af5f6
I20260810 21:03:32.761852 3574 ts_tablet_manager.cc:1939] T d08946839aa04f8d8017b5e8561af5f6 P 0e60debca709465c95416a933591bddf: Deleting consensus metadata
I20260810 21:03:32.761893 3576 ts_tablet_manager.cc:1905] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:32.762403 3031 catalog_manager.cc:4841] TS 0e60debca709465c95416a933591bddf (127.14.200.3:33717): tablet d08946839aa04f8d8017b5e8561af5f6 (table test-table [id=af348f0688f8402aa099a081994b8e43]) successfully deleted
I20260810 21:03:32.763492 3576 ts_tablet_manager.cc:1918] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:32.763628 3576 log.cc:1196] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/d08946839aa04f8d8017b5e8561af5f6
I20260810 21:03:32.763967 3576 ts_tablet_manager.cc:1939] T d08946839aa04f8d8017b5e8561af5f6 P 70da0c0dd4114a9f9460fdf291f9ae62: Deleting consensus metadata
I20260810 21:03:32.764457 3030 catalog_manager.cc:4841] TS 70da0c0dd4114a9f9460fdf291f9ae62 (127.14.200.2:43963): tablet d08946839aa04f8d8017b5e8561af5f6 (table test-table [id=af348f0688f8402aa099a081994b8e43]) successfully deleted
W20260810 21:03:32.768317 3029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:34783 (error 108)
I20260810 21:03:32.768837 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:32.768688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:34783 (local address 127.0.0.1:54348)
0810 21:03:32.768744 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:32.768804 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:03:32.768908 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3253
W20260810 21:03:32.773012 3029 catalog_manager.cc:4584] TS e58ac5ad71db458285011121201bf9d6 (127.14.200.1:34783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d08946839aa04f8d8017b5e8561af5f6: Network error: Client connection negotiation failed: client connection to 127.14.200.1:34783: connect: Connection refused (error 111)
I20260810 21:03:32.776980 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3394
I20260810 21:03:32.782482 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3012
2026-08-10T21:03:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (789 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:32 dist-test-slave-1x4g krb5kdc[3581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:32 dist-test-slave-1x4g krb5kdc[3581](info): set up 2 sockets
Aug 10 21:03:32 dist-test-slave-1x4g krb5kdc[3581](info): commencing operation
krb5kdc: starting...
W20260810 21:03:33.849961 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:33 dist-test-slave-1x4g krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:33.965822 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:46775
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:46775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:34.042721 3595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.042945 3595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.042997 3595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.044173 3595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:34.044240 3595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.044263 3595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.044284 3595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:34.044313 3595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:34.045683 3595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:46775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:46775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.045943 3595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.046159 3595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:34.050839 3595 server_base.cc:1034] running on GCE node
W20260810 21:03:34.050797 3601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.050810 3603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.050806 3600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.051316 3595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.051620 3595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.052811 3595 hybrid_clock.cc:648] HybridClock initialized: now 1786395814052807 us; error 35 us; skew 500 ppm
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:34.055111 3595 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:34.055526 3595 webserver.cc:460] Webserver started at http://127.14.200.62:43803/ using document root <none> and password file <none>
I20260810 21:03:34.055778 3595 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.055841 3595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.055977 3595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.057049 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "7518e3c3556f4963be9e63cf6f9ca930"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "662642d02212292644bd767fb09d0f55"
server_key_iv: "4fe28988011d0952f484efce779ca1ea"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.057402 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "7518e3c3556f4963be9e63cf6f9ca930"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "662642d02212292644bd767fb09d0f55"
server_key_iv: "4fe28988011d0952f484efce779ca1ea"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.058689 3595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:34.059334 3610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.059512 3595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:34.059625 3595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "7518e3c3556f4963be9e63cf6f9ca930"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "662642d02212292644bd767fb09d0f55"
server_key_iv: "4fe28988011d0952f484efce779ca1ea"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.059696 3595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.072382 3595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.074115 3595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.074283 3595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.078310 3595 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:46775
I20260810 21:03:34.078341 3664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:46775 every 8 connection(s)
I20260810 21:03:34.078696 3595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:34.079581 3665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:34.080019 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3595
I20260810 21:03:34.080148 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:34.080366 15136 external_mini_cluster.cc:1428] Setting key 4c0c68fa0838030c6e975c559ab7257f
I20260810 21:03:34.082692 3665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: Bootstrap starting.
I20260810 21:03:34.083276 3665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:34.083580 3665 log.cc:824] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:34.084254 3665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: No bootstrap required, opened a new log
I20260810 21:03:34.086261 3665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } }
I20260810 21:03:34.086442 3665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:34.086484 3665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7518e3c3556f4963be9e63cf6f9ca930, State: Initialized, Role: FOLLOWER
I20260810 21:03:34.086623 3665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } }
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:34.086702 3665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:34.086751 3665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:34.086802 3665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:34.087455 3665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } }
I20260810 21:03:34.087603 3665 leader_election.cc:302] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7518e3c3556f4963be9e63cf6f9ca930; no voters:
I20260810 21:03:34.087776 3665 leader_election.cc:288] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:34.087816 3672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:34.087961 3672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [term 1 LEADER]: Becoming Leader. State: Replica: 7518e3c3556f4963be9e63cf6f9ca930, State: Running, Role: LEADER
I20260810 21:03:34.088052 3665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:34.088070 3672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } }
I20260810 21:03:34.088456 3673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7518e3c3556f4963be9e63cf6f9ca930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } } }
I20260810 21:03:34.088598 3673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:34.089545 3674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7518e3c3556f4963be9e63cf6f9ca930. Latest consensus state: current_term: 1 leader_uuid: "7518e3c3556f4963be9e63cf6f9ca930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7518e3c3556f4963be9e63cf6f9ca930" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 46775 } } }
I20260810 21:03:34.089628 3674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930 [sys.catalog]: This master's current role is: LEADER
W20260810 21:03:34.089928 3687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:03:34.089990 3687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:03:34.090039 3681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:34.090233 3681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:34.091101 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.080869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.14.200.62:46775)
0810 21:03:34.081048 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.081056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.081080 (+ 24us) server_negotiation.cc:407] Connection header received
0810 21:03:34.081496 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.081503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.081546 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.081652 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.082413 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.083293 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.084285 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.084716 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.084840 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.087272 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.087276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.087276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:34.087285 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.089145 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.089589 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.089592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.089592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.089696 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.089933 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.089934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.089935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.090280 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.090425 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.090787 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.090950 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260810 21:03:34.092180 3681 catalog_manager.cc:1347] Generated new cluster ID: 7a730e72ccd54fb7b611d63159546071
I20260810 21:03:34.092231 3681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:34.098500 3681 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:34.098973 3681 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:34.103753 3681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: Generated new TSK 0
I20260810 21:03:34.103914 3681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.200.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
Entry for principal kudu/127.14.200.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab.
I20260810 21:03:34.137053 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:34.233599 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.233793 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.233815 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.234916 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.234958 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.234985 3693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:34.236483 3693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.1.keytab
--principal=kudu/127.14.200.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.236766 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.237071 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:34.241991 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.242034 3700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.242048 3702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.242252 3693 server_base.cc:1034] running on GCE node
I20260810 21:03:34.242406 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.242623 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.243796 3693 hybrid_clock.cc:648] HybridClock initialized: now 1786395814243786 us; error 29 us; skew 500 ppm
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:34.246186 3693 init.cc:377] Logged in from keytab as kudu/127.14.200.1@KRBTEST.COM (short username kudu)
I20260810 21:03:34.246652 3693 webserver.cc:460] Webserver started at http://127.14.200.1:38935/ using document root <none> and password file <none>
I20260810 21:03:34.246871 3693 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.246935 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.247087 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.248139 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "4281a0e5aa344af8b148ddb485d24b9a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "cefd9f7ec8069b691ea81fc32a4cbdaf"
server_key_iv: "60ca8fbc24d616d8270deca2b33ff282"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.248503 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "4281a0e5aa344af8b148ddb485d24b9a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "cefd9f7ec8069b691ea81fc32a4cbdaf"
server_key_iv: "60ca8fbc24d616d8270deca2b33ff282"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.249866 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:34.250597 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.250768 3693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:34.250867 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "4281a0e5aa344af8b148ddb485d24b9a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "cefd9f7ec8069b691ea81fc32a4cbdaf"
server_key_iv: "60ca8fbc24d616d8270deca2b33ff282"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.250952 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.269068 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.270792 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.270954 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.271196 3693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:34.271524 3693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:34.271574 3693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.271607 3693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:34.271631 3693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.277842 3693 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:42113
I20260810 21:03:34.277899 3831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:42113 every 8 connection(s)
I20260810 21:03:34.278244 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:34.282560 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3693
I20260810 21:03:34.282688 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:34.282960 15136 external_mini_cluster.cc:1428] Setting key e4d7b554e22cb143348235e900669785
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.1@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:34.288430 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.279032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:52821 (local address 127.14.200.62:46775)
0810 21:03:34.279100 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.279102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.279280 (+ 178us) server_negotiation.cc:407] Connection header received
0810 21:03:34.279775 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.279776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.279786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.279845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.280888 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.281389 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.282396 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.282626 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.282736 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.285481 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.285482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.285483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:34.285490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.286943 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.287369 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.287370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.287371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.287431 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.287687 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.287688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.287689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.287803 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.287874 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.288245 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.288339 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:03:34.288744 3834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.278964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46775 (local address 127.14.200.1:52821)
0810 21:03:34.279154 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.279197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.279684 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.279904 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.279907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.280523 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:34.280731 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.280735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.281468 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.281469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.282268 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.282270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.282816 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.282818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.282999 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.283639 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:34.283647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:34.285361 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:34.287052 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.287060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.287062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.287290 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.287500 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.287501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.287502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.287619 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.287876 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:34.287884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:34.288098 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.288211 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.288371 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260810 21:03:34.289238 3832 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46775
I20260810 21:03:34.289335 3832 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:34.289603 3832 heartbeater.cc:511] Master 127.14.200.62:46775 requested a full tablet report, sending...
I20260810 21:03:34.290145 3627 ts_manager.cc:194] Registered new tserver with Master: 4281a0e5aa344af8b148ddb485d24b9a (127.14.200.1:42113)
I20260810 21:03:34.290760 3627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.1@KRBTEST.COM'} at 127.14.200.1:52821
WARNING: no policy specified for kudu/127.14.200.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
Entry for principal kudu/127.14.200.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab.
I20260810 21:03:34.311846 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:34.394230 3837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.394403 3837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.394423 3837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.395407 3837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.395438 3837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.395462 3837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:34.396885 3837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.2.keytab
--principal=kudu/127.14.200.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.397058 3837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.397222 3837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:34.401938 3842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.401950 3846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.401970 3843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.402325 3837 server_base.cc:1034] running on GCE node
I20260810 21:03:34.402523 3837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.402797 3837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.403951 3837 hybrid_clock.cc:648] HybridClock initialized: now 1786395814403941 us; error 31 us; skew 500 ppm
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:34.406582 3837 init.cc:377] Logged in from keytab as kudu/127.14.200.2@KRBTEST.COM (short username kudu)
I20260810 21:03:34.407094 3837 webserver.cc:460] Webserver started at http://127.14.200.2:45597/ using document root <none> and password file <none>
I20260810 21:03:34.407339 3837 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.407400 3837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.407528 3837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.408465 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "2ad1bdedef144fa4975b847d3e548085"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "e7d760071fd173d8d5696fe23d56acca"
server_key_iv: "7fa2656135137e5846f024a320b1a689"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.408880 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "2ad1bdedef144fa4975b847d3e548085"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "e7d760071fd173d8d5696fe23d56acca"
server_key_iv: "7fa2656135137e5846f024a320b1a689"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.410181 3837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:34.410939 3853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.411175 3837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:34.411276 3837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "2ad1bdedef144fa4975b847d3e548085"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "e7d760071fd173d8d5696fe23d56acca"
server_key_iv: "7fa2656135137e5846f024a320b1a689"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.411362 3837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.423846 3837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.425958 3837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.426118 3837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.426337 3837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:34.426645 3837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:34.426712 3837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.426752 3837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:34.426780 3837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.432539 3837 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:35679
I20260810 21:03:34.432585 3973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:35679 every 8 connection(s)
I20260810 21:03:34.432920 3837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:03:34.437480 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3837
I20260810 21:03:34.437608 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:34.437830 15136 external_mini_cluster.cc:1428] Setting key cdfd4a2d35fb59f2ff4345c8177c86e0
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.2@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:34.442354 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.433582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46775 (local address 127.14.200.2:50227)
0810 21:03:34.433732 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.433771 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.434228 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.434423 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.434426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.434968 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:34.435147 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.435152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.435795 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.435796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.436547 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.436549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.436980 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.436981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.437081 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.437519 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:34.437530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:34.439335 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:34.441053 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.441060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.441061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.441270 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.441495 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.441495 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.441495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.441592 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.441872 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:34.441876 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:34.441982 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.442083 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.442200 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260810 21:03:34.442350 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.433608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:50227 (local address 127.14.200.62:46775)
0810 21:03:34.433680 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.433682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.433856 (+ 174us) server_negotiation.cc:407] Connection header received
0810 21:03:34.434312 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.434313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.434322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.434376 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.435222 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.435707 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.436629 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.436893 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.436949 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.439423 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.439425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.439425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:34.439432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.440970 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.441343 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.441344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.441344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.441415 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.441655 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.441656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.441657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.441804 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.441877 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.442130 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.442219 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 21:03:34.442864 3974 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46775
I20260810 21:03:34.442955 3974 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:34.443156 3974 heartbeater.cc:511] Master 127.14.200.62:46775 requested a full tablet report, sending...
I20260810 21:03:34.443550 3627 ts_manager.cc:194] Registered new tserver with Master: 2ad1bdedef144fa4975b847d3e548085 (127.14.200.2:35679)
I20260810 21:03:34.444178 3627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.2@KRBTEST.COM'} at 127.14.200.2:50227
WARNING: no policy specified for kudu/127.14.200.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
Entry for principal kudu/127.14.200.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab.
I20260810 21:03:34.464047 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:34.544514 3980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.544688 3980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.544726 3980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.545709 3980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.545746 3980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.545769 3980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:34.546983 3980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.3.keytab
--principal=kudu/127.14.200.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:46775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.547147 3980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.547298 3980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:34.551927 3986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.551995 3985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.551961 3988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.552251 3980 server_base.cc:1034] running on GCE node
I20260810 21:03:34.552443 3980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.552672 3980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.553813 3980 hybrid_clock.cc:648] HybridClock initialized: now 1786395814553800 us; error 52 us; skew 500 ppm
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:34.556106 3980 init.cc:377] Logged in from keytab as kudu/127.14.200.3@KRBTEST.COM (short username kudu)
I20260810 21:03:34.556537 3980 webserver.cc:460] Webserver started at http://127.14.200.3:43713/ using document root <none> and password file <none>
I20260810 21:03:34.556816 3980 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.556896 3980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.557031 3980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.557950 3980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "f40853b17fd4421dbb6ee24245ba7703"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "6b9e6c2ad06721e4997fd4d3b1365862"
server_key_iv: "3c76bf565f7fe900beff3b71730da252"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.558280 3980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "f40853b17fd4421dbb6ee24245ba7703"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "6b9e6c2ad06721e4997fd4d3b1365862"
server_key_iv: "3c76bf565f7fe900beff3b71730da252"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.559549 3980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:34.560241 3995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.560438 3980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:34.560541 3980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "f40853b17fd4421dbb6ee24245ba7703"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "6b9e6c2ad06721e4997fd4d3b1365862"
server_key_iv: "3c76bf565f7fe900beff3b71730da252"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.560626 3980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.571169 3980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.572767 3980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.572961 3980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.573151 3980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:34.573417 3980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:34.573467 3980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.573501 3980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:34.573527 3980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.579078 3980 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:39303
I20260810 21:03:34.579131 4115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:39303 every 8 connection(s)
I20260810 21:03:34.579440 3980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
Aug 10 21:03:34 dist-test-slave-1x4g krb5kdc[3581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.3@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:34.587500 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.580081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:44801 (local address 127.14.200.62:46775)
0810 21:03:34.580139 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.580141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.580326 (+ 185us) server_negotiation.cc:407] Connection header received
0810 21:03:34.580726 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.580727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.580736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.580833 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.581677 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.582121 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.582942 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.583108 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.583173 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.585123 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.585124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.585124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:34.585130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.586452 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.586780 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.586780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.586781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.586828 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.587039 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.587039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.587040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.587131 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.587180 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.587388 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.587442 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:03:34.587703 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.580038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:46775 (local address 127.14.200.3:44801)
0810 21:03:34.580185 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.580222 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.580649 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.580894 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.580897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.581417 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:34.581609 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.581613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.582195 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.582196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.582879 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.582881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.583194 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.583195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.583318 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.583687 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:03:34.583695 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:34.585043 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:34.586521 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.586526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.586528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.586718 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.586883 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:03:34.586884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:03:34.586884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:03:34.586986 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:03:34.587184 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:34.587188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:34.587286 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.587360 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.587481 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260810 21:03:34.588174 4116 heartbeater.cc:347] Connected to a master server at 127.14.200.62:46775
I20260810 21:03:34.588287 4116 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:34.588466 4116 heartbeater.cc:511] Master 127.14.200.62:46775 requested a full tablet report, sending...
I20260810 21:03:34.588821 3627 ts_manager.cc:194] Registered new tserver with Master: f40853b17fd4421dbb6ee24245ba7703 (127.14.200.3:39303)
I20260810 21:03:34.589044 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 3980
I20260810 21:03:34.589148 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:34.589323 15136 external_mini_cluster.cc:1428] Setting key 41b44600fa4d0bceb355fef99b1c7248
I20260810 21:03:34.589290 3627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.200.3@KRBTEST.COM'} at 127.14.200.3:44801
I20260810 21:03:34.590173 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:34.596750 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.591599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.14.200.62:46775)
0810 21:03:34.591630 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.591632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.591729 (+ 97us) server_negotiation.cc:407] Connection header received
0810 21:03:34.591773 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.591774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.591781 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.591818 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.592278 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.592693 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.593416 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.593584 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.593633 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.594578 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.594580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.594580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:34.594585 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.595860 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.596169 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.596170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.596170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.596216 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:34.596367 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:34.596368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:34.596368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:34.596481 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.596557 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.596645 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.596697 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 21:03:34.598629 3627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:34.600210 3627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:34.606609 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.14.200.1:42113)
0810 21:03:34.602847 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.602851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.602861 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:03:34.602904 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.602906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.602921 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.603014 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:34.603191 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.604239 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.605521 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.606212 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.606284 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.606341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.606495 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.606536 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260810 21:03:34.606621 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42113 (local address 127.0.0.1:36930)
0810 21:03:34.602751 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.602759 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.602795 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.603041 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.603045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.603048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:34.603107 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.603108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.604343 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.604344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.605432 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.605434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.606296 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.606296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.606374 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.606389 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.606478 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.606531 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":52,"thread_start_us":60,"threads_started":1}
I20260810 21:03:34.606865 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.14.200.3:39303)
0810 21:03:34.602950 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.602956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.602964 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:34.602995 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.602997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.603011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.603167 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:34.603361 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.603967 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.605434 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.606206 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.606284 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.606341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.606749 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.606788 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":89,"threads_started":1}
I20260810 21:03:34.606619 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39303 (local address 127.0.0.1:59540)
0810 21:03:34.602548 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.602596 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.602657 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.603234 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.603234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.603236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:34.603287 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.603288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.604074 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.604075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.605339 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.605342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.606294 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.606295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.606374 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.606389 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.606465 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.606532 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260810 21:03:34.607432 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:35679 (local address 127.0.0.1:54022)
0810 21:03:34.602612 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.602623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.602672 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.603080 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.603081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.603082 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:34.603118 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.603120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.604395 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.604396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.605348 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.605350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.607218 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.607219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.607301 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.607305 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.607335 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.607372 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":74,"threads_started":1}
I20260810 21:03:34.607367 3763 tablet_service.cc:1467] Processing CreateTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 (DEFAULT_TABLE table=test-table [id=a1e1ce9e732d410a92a96b3263fe2c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:34.607750 3763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:34.607838 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.602737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.14.200.2:35679)
0810 21:03:34.602917 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.602920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.602928 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:34.602959 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.602960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.602971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.603187 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:34.603258 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.604275 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.606377 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.607121 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.607627 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.607687 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.607723 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.607761 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260810 21:03:34.607749 4048 tablet_service.cc:1467] Processing CreateTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 (DEFAULT_TABLE table=test-table [id=a1e1ce9e732d410a92a96b3263fe2c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:34.608047 4048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:34.610210 4132 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: Bootstrap starting.
I20260810 21:03:34.610540 4133 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Bootstrap starting.
I20260810 21:03:34.610940 4132 tablet_bootstrap.cc:654] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:34.610884 3906 tablet_service.cc:1467] Processing CreateTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 (DEFAULT_TABLE table=test-table [id=a1e1ce9e732d410a92a96b3263fe2c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:34.611202 3906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:34.611305 4133 tablet_bootstrap.cc:654] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:34.611537 4132 log.cc:824] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:34.611868 4133 log.cc:824] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:34.612337 4132 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: No bootstrap required, opened a new log
I20260810 21:03:34.612406 4132 ts_tablet_manager.cc:1397] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:34.612650 4133 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: No bootstrap required, opened a new log
I20260810 21:03:34.612721 4133 ts_tablet_manager.cc:1397] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:34.613756 4132 raft_consensus.cc:348] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.613961 4132 raft_consensus.cc:374] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:34.614010 4132 raft_consensus.cc:729] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4281a0e5aa344af8b148ddb485d24b9a, State: Initialized, Role: FOLLOWER
I20260810 21:03:34.614131 4132 consensus_queue.cc:260] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.614421 4132 ts_tablet_manager.cc:1428] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:34.614537 3832 heartbeater.cc:503] Master 127.14.200.62:46775 was elected leader, sending a full tablet report...
I20260810 21:03:34.614598 4133 raft_consensus.cc:348] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.614755 4133 raft_consensus.cc:374] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:34.614796 4133 raft_consensus.cc:729] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40853b17fd4421dbb6ee24245ba7703, State: Initialized, Role: FOLLOWER
I20260810 21:03:34.614895 4133 consensus_queue.cc:260] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.615167 4133 ts_tablet_manager.cc:1428] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 21:03:34.615186 4116 heartbeater.cc:503] Master 127.14.200.62:46775 was elected leader, sending a full tablet report...
I20260810 21:03:34.615345 4137 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Bootstrap starting.
I20260810 21:03:34.615924 4137 tablet_bootstrap.cc:654] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:34.616394 4137 log.cc:824] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:34.617064 4137 tablet_bootstrap.cc:492] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: No bootstrap required, opened a new log
I20260810 21:03:34.617115 4137 ts_tablet_manager.cc:1397] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:03:34.618360 4137 raft_consensus.cc:348] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.618538 4137 raft_consensus.cc:374] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:34.618574 4137 raft_consensus.cc:729] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad1bdedef144fa4975b847d3e548085, State: Initialized, Role: FOLLOWER
I20260810 21:03:34.618731 4137 consensus_queue.cc:260] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.618816 4141 raft_consensus.cc:482] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:34.618904 4141 raft_consensus.cc:504] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.618985 4137 ts_tablet_manager.cc:1428] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:34.619004 3974 heartbeater.cc:503] Master 127.14.200.62:46775 was elected leader, sending a full tablet report...
I20260810 21:03:34.619248 4141 leader_election.cc:288] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f40853b17fd4421dbb6ee24245ba7703 (127.14.200.3:39303), 4281a0e5aa344af8b148ddb485d24b9a (127.14.200.1:42113)
I20260810 21:03:34.621968 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.619361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:39303 (local address 127.14.200.2:46667)
0810 21:03:34.619398 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.619409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.619450 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.619606 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.619607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.619608 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:34.619674 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.619676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.620367 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.620369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.621157 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.621159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.621705 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.621791 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.621793 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.621867 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.621906 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":9}
I20260810 21:03:34.621963 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.619461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:46667 (local address 127.14.200.3:39303)
0810 21:03:34.619495 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.619497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.619505 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:03:34.619526 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.619527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.619535 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.619599 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:34.619761 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.620264 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621244 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.621628 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621706 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.621714 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.621876 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.621908 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:03:34.622080 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.619524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:53299 (local address 127.14.200.1:42113)
0810 21:03:34.619568 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.619570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.619719 (+ 149us) server_negotiation.cc:407] Connection header received
0810 21:03:34.619777 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.619778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.619788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.619840 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:34.619976 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.620410 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621369 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.621747 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621836 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.621844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.621985 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.622017 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:03:34.622140 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.619361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42113 (local address 127.14.200.2:53299)
0810 21:03:34.619653 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.619662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.619730 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.619844 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.619845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.619846 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:34.619907 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.619908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.620496 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.620497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.621284 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.621286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.621840 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.621841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.621909 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.621913 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.621980 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.622062 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":69,"threads_started":1}
I20260810 21:03:34.622257 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3dac2ed0ba42faafe7687f5ad7b9a8" candidate_uuid: "2ad1bdedef144fa4975b847d3e548085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40853b17fd4421dbb6ee24245ba7703" is_pre_election: true
I20260810 21:03:34.622371 4069 raft_consensus.cc:2454] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ad1bdedef144fa4975b847d3e548085 in term 0.
I20260810 21:03:34.622450 3785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3dac2ed0ba42faafe7687f5ad7b9a8" candidate_uuid: "2ad1bdedef144fa4975b847d3e548085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4281a0e5aa344af8b148ddb485d24b9a" is_pre_election: true
I20260810 21:03:34.622565 3785 raft_consensus.cc:2454] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ad1bdedef144fa4975b847d3e548085 in term 0.
I20260810 21:03:34.622591 3856 leader_election.cc:302] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad1bdedef144fa4975b847d3e548085, f40853b17fd4421dbb6ee24245ba7703; no voters:
I20260810 21:03:34.622709 4141 raft_consensus.cc:2781] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:34.622772 4141 raft_consensus.cc:482] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:34.622803 4141 raft_consensus.cc:3037] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:34.623378 4141 raft_consensus.cc:504] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.623669 3785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3dac2ed0ba42faafe7687f5ad7b9a8" candidate_uuid: "2ad1bdedef144fa4975b847d3e548085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4281a0e5aa344af8b148ddb485d24b9a"
I20260810 21:03:34.623684 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3dac2ed0ba42faafe7687f5ad7b9a8" candidate_uuid: "2ad1bdedef144fa4975b847d3e548085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40853b17fd4421dbb6ee24245ba7703"
I20260810 21:03:34.623752 4069 raft_consensus.cc:3037] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:34.623762 3785 raft_consensus.cc:3037] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:34.624329 3785 raft_consensus.cc:2454] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ad1bdedef144fa4975b847d3e548085 in term 1.
I20260810 21:03:34.624333 4069 raft_consensus.cc:2454] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ad1bdedef144fa4975b847d3e548085 in term 1.
I20260810 21:03:34.623540 4141 leader_election.cc:288] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [CANDIDATE]: Term 1 election: Requested vote from peers f40853b17fd4421dbb6ee24245ba7703 (127.14.200.3:39303), 4281a0e5aa344af8b148ddb485d24b9a (127.14.200.1:42113)
I20260810 21:03:34.624482 3856 leader_election.cc:302] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad1bdedef144fa4975b847d3e548085, f40853b17fd4421dbb6ee24245ba7703; no voters:
I20260810 21:03:34.624569 4141 raft_consensus.cc:2781] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:34.624624 4141 raft_consensus.cc:686] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad1bdedef144fa4975b847d3e548085, State: Running, Role: LEADER
I20260810 21:03:34.624693 4141 consensus_queue.cc:237] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } }
I20260810 21:03:34.625414 3626 catalog_manager.cc:5495] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ad1bdedef144fa4975b847d3e548085 (127.14.200.2). New cstate: current_term: 1 leader_uuid: "2ad1bdedef144fa4975b847d3e548085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ad1bdedef144fa4975b847d3e548085" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 35679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 } health_report { overall_health: UNKNOWN } } }
I20260810 21:03:34.634030 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.631706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.14.200.2:35679)
0810 21:03:34.631746 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.631748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.631757 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:34.631833 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.631833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.631849 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.631934 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:34.632042 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.632494 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.633315 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.633491 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.633559 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.633716 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:34.633803 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:34.633883 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.633932 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.633949 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:34.635447 4069 raft_consensus.cc:1261] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 1 FOLLOWER]: Refusing update from remote peer 2ad1bdedef144fa4975b847d3e548085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:34.635466 3785 raft_consensus.cc:1261] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a [term 1 FOLLOWER]: Refusing update from remote peer 2ad1bdedef144fa4975b847d3e548085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:34.635828 4141 consensus_queue.cc:1035] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40853b17fd4421dbb6ee24245ba7703" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:34.635986 4142 consensus_queue.cc:1035] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4281a0e5aa344af8b148ddb485d24b9a" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:34.638543 4150 mvcc.cc:205] Tried to move back new op lower bound from 7317077256744804352 to 7317077256703610880. Current Snapshot: MvccSnapshot[applied={T|T < 7317077256744804352}]
I20260810 21:03:34.640090 3626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:34.640193 3626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:34.640878 3626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7518e3c3556f4963be9e63cf6f9ca930: Sending DeleteTablet for 3 replicas of tablet ee3dac2ed0ba42faafe7687f5ad7b9a8
I20260810 21:03:34.641194 4048 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:34 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:59540
I20260810 21:03:34.641233 3906 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:34 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:54022
I20260810 21:03:34.641484 3763 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:34 UTC) from {username='kudu', principal='kudu/127.14.200.62@KRBTEST.COM'} at 127.0.0.1:36930
I20260810 21:03:34.641671 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3693
I20260810 21:03:34.641685 4160 tablet_replica.cc:330] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 4281a0e5aa344af8b148ddb485d24b9a: stopping tablet replica
I20260810 21:03:34.648885 4159 tablet_replica.cc:330] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: stopping tablet replica
I20260810 21:03:34.648990 4159 raft_consensus.cc:2229] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:34.649098 4159 raft_consensus.cc:2258] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:34.649255 4158 tablet_replica.cc:330] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: stopping tablet replica
I20260810 21:03:34.649454 4158 raft_consensus.cc:2229] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:34.649561 4158 raft_consensus.cc:2258] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:34.649585 4159 ts_tablet_manager.cc:1905] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:34.649956 4158 ts_tablet_manager.cc:1905] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:34.651364 4159 ts_tablet_manager.cc:1918] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:34.651527 4159 log.cc:1196] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/ee3dac2ed0ba42faafe7687f5ad7b9a8
I20260810 21:03:34.651544 4158 ts_tablet_manager.cc:1918] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:34.651604 4158 log.cc:1196] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/ee3dac2ed0ba42faafe7687f5ad7b9a8
I20260810 21:03:34.651872 4158 ts_tablet_manager.cc:1939] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P f40853b17fd4421dbb6ee24245ba7703: Deleting consensus metadata
I20260810 21:03:34.651952 4159 ts_tablet_manager.cc:1939] T ee3dac2ed0ba42faafe7687f5ad7b9a8 P 2ad1bdedef144fa4975b847d3e548085: Deleting consensus metadata
W20260810 21:03:34.652136 3611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:42113 (error 108)
I20260810 21:03:34.652522 3614 catalog_manager.cc:4841] TS 2ad1bdedef144fa4975b847d3e548085 (127.14.200.2:35679): tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 (table test-table [id=a1e1ce9e732d410a92a96b3263fe2c58]) successfully deleted
I20260810 21:03:34.652616 3613 catalog_manager.cc:4841] TS f40853b17fd4421dbb6ee24245ba7703 (127.14.200.3:39303): tablet ee3dac2ed0ba42faafe7687f5ad7b9a8 (table test-table [id=a1e1ce9e732d410a92a96b3263fe2c58]) successfully deleted
I20260810 21:03:34.652772 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.652672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:42113 (local address 127.0.0.1:36940)
0810 21:03:34.652711 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.652743 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 21:03:34.652906 3611 catalog_manager.cc:4584] TS 4281a0e5aa344af8b148ddb485d24b9a (127.14.200.1:42113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee3dac2ed0ba42faafe7687f5ad7b9a8: Network error: Client connection negotiation failed: client connection to 127.14.200.1:42113: connect: Connection refused (error 111)
I20260810 21:03:34.652948 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3837
I20260810 21:03:34.660022 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3980
I20260810 21:03:34.667551 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 3595
2026-08-10T21:03:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1885 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T21:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:34Z Disabled control of system clock
I20260810 21:03:34.696138 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39097
--webserver_interface=127.14.200.62
--webserver_port=0
--builtin_ntp_servers=127.14.200.20:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:34.773458 4165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.773641 4165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.773659 4165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.774600 4165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:34.774631 4165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.774642 4165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.774653 4165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:34.774664 4165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:34.775815 4165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.775995 4165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.776229 4165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:34.780864 4172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.780905 4165 server_base.cc:1034] running on GCE node
W20260810 21:03:34.780881 4171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.780871 4174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.781270 4165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.781566 4165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.782724 4165 hybrid_clock.cc:648] HybridClock initialized: now 1786395814782713 us; error 45 us; skew 500 ppm
I20260810 21:03:34.783844 4165 webserver.cc:460] Webserver started at http://127.14.200.62:40197/ using document root <none> and password file <none>
I20260810 21:03:34.784089 4165 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.784157 4165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.784304 4165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.785400 4165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "0940ce7f8c7e46149fad3278aa84cc5a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "08daa3d5c0007521e2ed2ff184c6d04d"
server_key_iv: "e2dceca345295ee2f1331774a97b48de"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.785772 4165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "0940ce7f8c7e46149fad3278aa84cc5a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "08daa3d5c0007521e2ed2ff184c6d04d"
server_key_iv: "e2dceca345295ee2f1331774a97b48de"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.786995 4165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:34.787644 4180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.787788 4165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:34.787869 4165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "0940ce7f8c7e46149fad3278aa84cc5a"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "08daa3d5c0007521e2ed2ff184c6d04d"
server_key_iv: "e2dceca345295ee2f1331774a97b48de"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.787921 4165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.800376 4165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.800591 4165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.800683 4165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.804221 4165 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:39097
I20260810 21:03:34.804282 4239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:39097 every 8 connection(s)
I20260810 21:03:34.804579 4165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:34.805555 4240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:34.807588 4240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: Bootstrap starting.
I20260810 21:03:34.808127 4240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:34.808410 4240 log.cc:824] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:34.809115 4240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: No bootstrap required, opened a new log
I20260810 21:03:34.810472 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4165
I20260810 21:03:34.810600 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:34.810791 15136 external_mini_cluster.cc:1428] Setting key 22f089ffea2a5f0bc8c705dbaeecfa67
I20260810 21:03:34.810989 4240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } }
I20260810 21:03:34.811142 4240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:34.811184 4240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0940ce7f8c7e46149fad3278aa84cc5a, State: Initialized, Role: FOLLOWER
I20260810 21:03:34.811314 4240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } }
I20260810 21:03:34.811403 4240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:34.811442 4240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:34.811498 4240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:34.812098 4240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } }
I20260810 21:03:34.812199 4240 leader_election.cc:302] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0940ce7f8c7e46149fad3278aa84cc5a; no voters:
I20260810 21:03:34.812371 4240 leader_election.cc:288] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:34.812438 4245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:34.812620 4240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:34.812629 4245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [term 1 LEADER]: Becoming Leader. State: Replica: 0940ce7f8c7e46149fad3278aa84cc5a, State: Running, Role: LEADER
I20260810 21:03:34.812745 4245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } }
I20260810 21:03:34.813275 4247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0940ce7f8c7e46149fad3278aa84cc5a. Latest consensus state: current_term: 1 leader_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } } }
I20260810 21:03:34.813491 4247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:34.813427 4246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0940ce7f8c7e46149fad3278aa84cc5a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39097 } } }
I20260810 21:03:34.813774 4246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:34.813979 4253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:34.814952 4253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:34.816287 4253 catalog_manager.cc:1347] Generated new cluster ID: b017ebed8de1490a8b1bfea89af82d32
I20260810 21:03:34.816361 4253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:34.817375 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.811249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.14.200.62:39097)
0810 21:03:34.811436 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.811457 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.811489 (+ 32us) server_negotiation.cc:407] Connection header received
0810 21:03:34.811720 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.811725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.811781 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.811881 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.812111 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.813797 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.814985 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.815569 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.815681 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.816511 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.816514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.816515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:34.816739 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.816779 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:34.816817 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.816877 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.817098 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.817172 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260810 21:03:34.822721 4253 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:34.823263 4253 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:34.829087 4253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: Generated new TSK 0
I20260810 21:03:34.829250 4253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:34.830294 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.1:0
--local_ip_for_outbound_sockets=127.14.200.1
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--builtin_ntp_servers=127.14.200.20:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:34.910988 4265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:34.911191 4265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:34.911213 4265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:34.912336 4265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:34.912372 4265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:34.912395 4265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.1
I20260810 21:03:34.913836 4265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.200.1
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.1
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:34.914088 4265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:34.914419 4265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:34.920085 4271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.920114 4273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:34.920076 4270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:34.920501 4265 server_base.cc:1034] running on GCE node
I20260810 21:03:34.920710 4265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:34.921003 4265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:34.922184 4265 hybrid_clock.cc:648] HybridClock initialized: now 1786395814922171 us; error 27 us; skew 500 ppm
I20260810 21:03:34.923504 4265 webserver.cc:460] Webserver started at http://127.14.200.1:38935/ using document root <none> and password file <none>
I20260810 21:03:34.923753 4265 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:34.923805 4265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:34.923925 4265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:34.925180 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data/instance:
uuid: "02c13bb02a3342dfb5d71af0fc000136"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "202dc620e50f4e0a74d40e11c3c6ce25"
server_key_iv: "8a4f0b4b7ceff36677a69be941fe8b21"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.925536 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance:
uuid: "02c13bb02a3342dfb5d71af0fc000136"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "202dc620e50f4e0a74d40e11c3c6ce25"
server_key_iv: "8a4f0b4b7ceff36677a69be941fe8b21"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.927057 4265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:34.927803 4279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.928005 4265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:34.928097 4265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal
uuid: "02c13bb02a3342dfb5d71af0fc000136"
format_stamp: "Formatted at 2026-08-10 21:03:34 on dist-test-slave-1x4g"
server_key: "202dc620e50f4e0a74d40e11c3c6ce25"
server_key_iv: "8a4f0b4b7ceff36677a69be941fe8b21"
server_key_version: "encryptionkey@0"
I20260810 21:03:34.928159 4265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:34.939215 4265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:34.939482 4265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:34.939603 4265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:34.939800 4265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:34.940107 4265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:34.940145 4265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.940171 4265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:34.940193 4265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:34.947013 4265 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.1:41747
I20260810 21:03:34.947065 4402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.1:41747 every 8 connection(s)
I20260810 21:03:34.947283 4265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/data/info.pb
I20260810 21:03:34.951836 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.947888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.1:38945 (local address 127.14.200.62:39097)
0810 21:03:34.947929 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:34.947931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:34.948162 (+ 231us) server_negotiation.cc:407] Connection header received
0810 21:03:34.948609 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:34.948610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:34.948619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:34.948666 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:34.949325 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.949763 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.950714 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.950892 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.950943 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.951637 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:34.951639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:34.951639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:34.951643 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:34.951655 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:34.951658 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:34.951698 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:34.951757 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 21:03:34.951782 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:03:34.952030 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:34.947895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39097 (local address 127.14.200.1:38945)
0810 21:03:34.948053 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:34.948082 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:34.948534 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:34.948730 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:34.948733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:34.948740 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:34.949258 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.949263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.949837 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.949839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.950638 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:34.950641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:34.950975 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:34.950976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:34.951079 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:34.951506 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:34.951507 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:34.951532 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:34.951552 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:34.951699 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:34.951701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:34.951703 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:34.951745 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 21:03:34.951813 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260810 21:03:34.952553 4403 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39097
I20260810 21:03:34.952657 4403 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:34.952890 4403 heartbeater.cc:511] Master 127.14.200.62:39097 requested a full tablet report, sending...
I20260810 21:03:34.953394 4197 ts_manager.cc:194] Registered new tserver with Master: 02c13bb02a3342dfb5d71af0fc000136 (127.14.200.1:41747)
I20260810 21:03:34.953959 4197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.1:38945
I20260810 21:03:34.956636 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4265
I20260810 21:03:34.956768 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-0/wal/instance
I20260810 21:03:34.956974 15136 external_mini_cluster.cc:1428] Setting key 0a07ec0acf2564205efe243be9ece40f
I20260810 21:03:34.957907 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.2:0
--local_ip_for_outbound_sockets=127.14.200.2
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--builtin_ntp_servers=127.14.200.20:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 21:03:35.035863 4406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:35.036038 4406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:35.036057 4406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:35.037195 4406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:35.037232 4406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:35.037256 4406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.2
I20260810 21:03:35.038546 4406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.200.2
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.2
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:35.038769 4406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:35.039116 4406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:35.044508 4415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:35.044498 4413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:35.044502 4412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:35.044533 4406 server_base.cc:1034] running on GCE node
I20260810 21:03:35.044901 4406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:35.045133 4406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:35.046317 4406 hybrid_clock.cc:648] HybridClock initialized: now 1786395815046303 us; error 26 us; skew 500 ppm
I20260810 21:03:35.047632 4406 webserver.cc:460] Webserver started at http://127.14.200.2:33451/ using document root <none> and password file <none>
I20260810 21:03:35.047885 4406 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:35.047953 4406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:35.048098 4406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:35.049144 4406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data/instance:
uuid: "11c2bc2da8be4e3aad13f70b92b75f83"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "4754825191d0164e94bd010ee75ed9f2"
server_key_iv: "797a44032521af6cf0083967ab36e536"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.049486 4406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance:
uuid: "11c2bc2da8be4e3aad13f70b92b75f83"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "4754825191d0164e94bd010ee75ed9f2"
server_key_iv: "797a44032521af6cf0083967ab36e536"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.050841 4406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:35.051606 4422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.051764 4406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:35.051858 4406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal
uuid: "11c2bc2da8be4e3aad13f70b92b75f83"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "4754825191d0164e94bd010ee75ed9f2"
server_key_iv: "797a44032521af6cf0083967ab36e536"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.051929 4406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:35.063834 4406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:35.064082 4406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:35.064215 4406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:35.064411 4406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:35.064731 4406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:35.064783 4406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.064858 4406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:35.064886 4406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.070427 4406 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.2:39285
I20260810 21:03:35.070474 4549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.2:39285 every 8 connection(s)
I20260810 21:03:35.070765 4406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/data/info.pb
I20260810 21:03:35.073272 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4406
I20260810 21:03:35.073390 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal/instance
I20260810 21:03:35.073562 15136 external_mini_cluster.cc:1428] Setting key 6d7ea87bbbfa3c64be972b24cd74f3d8
I20260810 21:03:35.074510 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.200.3:0
--local_ip_for_outbound_sockets=127.14.200.3
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--builtin_ntp_servers=127.14.200.20:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 21:03:35.076222 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.071440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.2:53009 (local address 127.14.200.62:39097)
0810 21:03:35.071499 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.071501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.071729 (+ 228us) server_negotiation.cc:407] Connection header received
0810 21:03:35.072222 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.072224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.072239 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.072290 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:35.073030 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.073586 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.074724 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.075035 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.075113 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.075899 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:35.075901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:35.075901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:35.075906 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:35.075923 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:35.075928 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:35.075990 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.076078 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.076111 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:03:35.076501 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.071416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39097 (local address 127.14.200.2:53009)
0810 21:03:35.071613 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.071643 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.072120 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.072351 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.072355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.072364 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:35.072937 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.072943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.073680 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.073681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.074613 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.074617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.075125 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.075126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.075230 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.075766 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:35.075767 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:35.075793 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:35.075814 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:35.075993 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:35.075996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:35.075998 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.076079 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.076197 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260810 21:03:35.077076 4550 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39097
I20260810 21:03:35.077189 4550 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:35.077363 4550 heartbeater.cc:511] Master 127.14.200.62:39097 requested a full tablet report, sending...
I20260810 21:03:35.077783 4197 ts_manager.cc:194] Registered new tserver with Master: 11c2bc2da8be4e3aad13f70b92b75f83 (127.14.200.2:39285)
I20260810 21:03:35.078258 4197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.2:53009
W20260810 21:03:35.155874 4553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:35.156060 4553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:35.156080 4553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:35.157116 4553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:35.157148 4553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:35.157172 4553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.200.3
I20260810 21:03:35.158357 4553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.200.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.200.3
--webserver_port=0
--tserver_master_addrs=127.14.200.62:39097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.200.3
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:35.158517 4553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:35.158723 4553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:35.163558 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:35.163581 4561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:35.163622 4553 server_base.cc:1034] running on GCE node
W20260810 21:03:35.163586 4560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:35.163938 4553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:35.164171 4553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:35.165337 4553 hybrid_clock.cc:648] HybridClock initialized: now 1786395815165317 us; error 34 us; skew 500 ppm
I20260810 21:03:35.166486 4553 webserver.cc:460] Webserver started at http://127.14.200.3:40777/ using document root <none> and password file <none>
I20260810 21:03:35.166754 4553 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:35.166818 4553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:35.166970 4553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:35.167963 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data/instance:
uuid: "a15373ca1a9d4a3499c075558900f6f5"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "10437256ea711fca09efb380c1d24842"
server_key_iv: "dde05071ac4ecf0e69290da2a0f6f2ca"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.168313 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance:
uuid: "a15373ca1a9d4a3499c075558900f6f5"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "10437256ea711fca09efb380c1d24842"
server_key_iv: "dde05071ac4ecf0e69290da2a0f6f2ca"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.169708 4553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:35.170435 4571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.170609 4553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:35.170708 4553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal
uuid: "a15373ca1a9d4a3499c075558900f6f5"
format_stamp: "Formatted at 2026-08-10 21:03:35 on dist-test-slave-1x4g"
server_key: "10437256ea711fca09efb380c1d24842"
server_key_iv: "dde05071ac4ecf0e69290da2a0f6f2ca"
server_key_version: "encryptionkey@0"
I20260810 21:03:35.170787 4553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:35.219035 4553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:35.219491 4553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:35.219653 4553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:35.219960 4553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 21:03:35.220363 4553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 21:03:35.220424 4553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.220467 4553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 21:03:35.220496 4553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:35.226307 4553 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.3:40251
I20260810 21:03:35.226367 4693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.3:40251 every 8 connection(s)
I20260810 21:03:35.226819 4553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/data/info.pb
I20260810 21:03:35.231181 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4553
I20260810 21:03:35.231318 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal/instance
I20260810 21:03:35.231573 15136 external_mini_cluster.cc:1428] Setting key 3a69587cc05b35e023c599aaebf86268
I20260810 21:03:35.232276 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.227651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:38079 (local address 127.14.200.62:39097)
0810 21:03:35.227740 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.227742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.227938 (+ 196us) server_negotiation.cc:407] Connection header received
0810 21:03:35.228470 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.228471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.228482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.228541 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:35.229244 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.229726 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.230658 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.230836 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.230887 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.231947 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:35.231948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:35.231948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:35.231952 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:35.231965 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:35.231969 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:35.232056 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.232159 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.232189 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 21:03:35.232522 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.227610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:39097 (local address 127.14.200.3:38079)
0810 21:03:35.227816 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.227851 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.228341 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.228625 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.228628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.228636 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:03:35.229164 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.229169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.229808 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.229809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.230584 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.230585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.230912 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.230913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.231026 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.231708 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 21:03:35.231708 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:03:35.231764 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 21:03:35.231796 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:03:35.232058 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:03:35.232066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:03:35.232070 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.232159 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.232288 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260810 21:03:35.233018 4694 heartbeater.cc:347] Connected to a master server at 127.14.200.62:39097
I20260810 21:03:35.233096 4694 heartbeater.cc:464] Registering TS with master...
I20260810 21:03:35.233271 4694 heartbeater.cc:511] Master 127.14.200.62:39097 requested a full tablet report, sending...
I20260810 21:03:35.233623 4197 ts_manager.cc:194] Registered new tserver with Master: a15373ca1a9d4a3499c075558900f6f5 (127.14.200.3:40251)
I20260810 21:03:35.233942 4197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.200.3:38079
I20260810 21:03:35.243384 15136 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 21:03:35.247784 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.244896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.14.200.62:39097)
0810 21:03:35.244953 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.244955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.245061 (+ 106us) server_negotiation.cc:407] Connection header received
0810 21:03:35.245120 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.245121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.245166 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.245281 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:35.245416 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.245865 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.246659 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.246867 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.246949 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.247541 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:35.247542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:35.247543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:35.247547 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:35.247560 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:35.247564 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:35.247607 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.247663 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.247685 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:03:35.249264 4197 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 21:03:35.250785 4197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 21:03:35.257112 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.252946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:39285 (local address 127.0.0.1:38400)
0810 21:03:35.253385 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.253428 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.253505 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.253801 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.253801 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.253802 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:35.253866 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.253869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.255168 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.255169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.255952 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.255954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.256811 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.256812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.256890 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.256903 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.256981 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.257031 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":12,"thread_start_us":87,"threads_started":1}
I20260810 21:03:35.257158 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.253175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.14.200.2:39285)
0810 21:03:35.253386 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.253390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.253533 (+ 143us) server_negotiation.cc:407] Connection header received
0810 21:03:35.253657 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.253659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.253674 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.253736 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:35.253964 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.255072 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.256042 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.256689 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.256764 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.256872 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.257009 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.257061 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260810 21:03:35.257510 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.252961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41747 (local address 127.0.0.1:40848)
0810 21:03:35.253625 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.253633 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.253681 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.253904 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.253905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.253907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:35.253967 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.253969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.254862 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.254863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.256346 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.256348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.257290 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.257291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.257345 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.257348 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.257411 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.257449 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":28,"thread_start_us":74,"threads_started":1}
I20260810 21:03:35.257572 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.253171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.14.200.1:41747)
0810 21:03:35.253376 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.253380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.253737 (+ 357us) server_negotiation.cc:407] Connection header received
0810 21:03:35.253783 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.253785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.253838 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.253914 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:35.254021 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.254779 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.256447 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.257097 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.257161 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.257255 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.257447 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.257482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260810 21:03:35.257822 4478 tablet_service.cc:1467] Processing CreateTablet for tablet 21fec8daeec44df58a98ea745cdc8547 (DEFAULT_TABLE table=test-table [id=2c4da92b03744ea281ad2facabd43a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:35.258183 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.253009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.14.200.3:40251)
0810 21:03:35.253173 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.253177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.253508 (+ 331us) server_negotiation.cc:407] Connection header received
0810 21:03:35.253623 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.253625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.253641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.253724 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:35.253963 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.255053 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.256312 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.257068 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.257133 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.257796 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.258062 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.258097 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260810 21:03:35.258205 4478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fec8daeec44df58a98ea745cdc8547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:35.258368 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.252940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.3:40251 (local address 127.0.0.1:46576)
0810 21:03:35.253385 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.253428 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.253505 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.253796 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.253799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.253801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:35.253866 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.253868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.255153 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.255154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.256224 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.256227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.257847 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.257848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.257906 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.257909 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.257963 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.258307 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":29,"thread_start_us":52,"threads_started":1}
I20260810 21:03:35.259034 4624 tablet_service.cc:1467] Processing CreateTablet for tablet 21fec8daeec44df58a98ea745cdc8547 (DEFAULT_TABLE table=test-table [id=2c4da92b03744ea281ad2facabd43a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:35.259354 4624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fec8daeec44df58a98ea745cdc8547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:35.261636 4713 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Bootstrap starting.
I20260810 21:03:35.261942 4333 tablet_service.cc:1467] Processing CreateTablet for tablet 21fec8daeec44df58a98ea745cdc8547 (DEFAULT_TABLE table=test-table [id=2c4da92b03744ea281ad2facabd43a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 21:03:35.262331 4713 tablet_bootstrap.cc:654] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:35.262327 4333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21fec8daeec44df58a98ea745cdc8547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:35.262692 4713 log.cc:824] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:35.262727 4714 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Bootstrap starting.
I20260810 21:03:35.263473 4714 tablet_bootstrap.cc:654] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:35.263554 4713 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: No bootstrap required, opened a new log
I20260810 21:03:35.263623 4713 ts_tablet_manager.cc:1397] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 21:03:35.263873 4714 log.cc:824] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:35.265193 4717 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: Bootstrap starting.
I20260810 21:03:35.265667 4714 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: No bootstrap required, opened a new log
I20260810 21:03:35.265766 4714 ts_tablet_manager.cc:1397] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 21:03:35.265827 4717 tablet_bootstrap.cc:654] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:35.266175 4717 log.cc:824] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:35.266338 4713 raft_consensus.cc:348] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.266511 4713 raft_consensus.cc:374] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:35.266554 4713 raft_consensus.cc:729] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c2bc2da8be4e3aad13f70b92b75f83, State: Initialized, Role: FOLLOWER
I20260810 21:03:35.266697 4713 consensus_queue.cc:260] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.266938 4717 tablet_bootstrap.cc:492] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: No bootstrap required, opened a new log
I20260810 21:03:35.266961 4713 ts_tablet_manager.cc:1428] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 21:03:35.267002 4717 ts_tablet_manager.cc:1397] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 21:03:35.267069 4550 heartbeater.cc:503] Master 127.14.200.62:39097 was elected leader, sending a full tablet report...
I20260810 21:03:35.268074 4714 raft_consensus.cc:348] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.268229 4714 raft_consensus.cc:374] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:35.268270 4714 raft_consensus.cc:729] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15373ca1a9d4a3499c075558900f6f5, State: Initialized, Role: FOLLOWER
I20260810 21:03:35.268401 4714 consensus_queue.cc:260] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.268672 4714 ts_tablet_manager.cc:1428] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:03:35.268718 4694 heartbeater.cc:503] Master 127.14.200.62:39097 was elected leader, sending a full tablet report...
I20260810 21:03:35.269423 4717 raft_consensus.cc:348] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.269596 4717 raft_consensus.cc:374] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:35.269647 4717 raft_consensus.cc:729] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c13bb02a3342dfb5d71af0fc000136, State: Initialized, Role: FOLLOWER
I20260810 21:03:35.269752 4717 consensus_queue.cc:260] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.270011 4717 ts_tablet_manager.cc:1428] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 21:03:35.270043 4403 heartbeater.cc:503] Master 127.14.200.62:39097 was elected leader, sending a full tablet report...
W20260810 21:03:35.321278 4551 tablet.cc:2369] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 21:03:35.417150 4720 raft_consensus.cc:482] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 21:03:35.417273 4720 raft_consensus.cc:504] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.417675 4720 leader_election.cc:288] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11c2bc2da8be4e3aad13f70b92b75f83 (127.14.200.2:39285), 02c13bb02a3342dfb5d71af0fc000136 (127.14.200.1:41747)
I20260810 21:03:35.421541 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.417980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:47669 (local address 127.14.200.2:39285)
0810 21:03:35.418057 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.418059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.418071 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:03:35.418104 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.418105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.418119 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.418184 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:35.418376 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.419142 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.420548 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.421159 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.421237 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.421245 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.421425 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.421468 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 21:03:35.421541 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.417992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.200.3:32769 (local address 127.14.200.1:41747)
0810 21:03:35.418058 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.418060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.418310 (+ 250us) server_negotiation.cc:407] Connection header received
0810 21:03:35.418371 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.418372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.418385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.418443 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 21:03:35.418588 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.419236 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.420549 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.421159 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.421237 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.421245 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.421425 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.421470 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:03:35.421546 4696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.417874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.2:39285 (local address 127.14.200.3:47669)
0810 21:03:35.417939 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.417953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.418025 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.418208 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.418209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.418211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:35.418291 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.418294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.419219 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.419219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.420451 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.420455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.421247 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.421248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.421336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.421342 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.421401 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.421452 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":15}
I20260810 21:03:35.421554 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.417874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41747 (local address 127.14.200.3:32769)
0810 21:03:35.418234 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.418244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:03:35.418300 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:03:35.418448 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:03:35.418449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:03:35.418450 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 21:03:35.418503 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.418504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.419311 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.419312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.420452 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:03:35.420455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.421247 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.421248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:03:35.421336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.421342 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 21:03:35.421401 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 21:03:35.421451 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":170,"thread_start_us":113,"threads_started":1}
I20260810 21:03:35.421957 4354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fec8daeec44df58a98ea745cdc8547" candidate_uuid: "a15373ca1a9d4a3499c075558900f6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c13bb02a3342dfb5d71af0fc000136" is_pre_election: true
I20260810 21:03:35.421939 4499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fec8daeec44df58a98ea745cdc8547" candidate_uuid: "a15373ca1a9d4a3499c075558900f6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" is_pre_election: true
I20260810 21:03:35.422108 4354 raft_consensus.cc:2454] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a15373ca1a9d4a3499c075558900f6f5 in term 0.
I20260810 21:03:35.422120 4499 raft_consensus.cc:2454] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a15373ca1a9d4a3499c075558900f6f5 in term 0.
I20260810 21:03:35.422355 4575 leader_election.cc:302] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c13bb02a3342dfb5d71af0fc000136, a15373ca1a9d4a3499c075558900f6f5; no voters:
I20260810 21:03:35.422498 4720 raft_consensus.cc:2781] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 21:03:35.422551 4720 raft_consensus.cc:482] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 21:03:35.422573 4720 raft_consensus.cc:3037] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:35.423270 4720 raft_consensus.cc:504] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.423417 4720 leader_election.cc:288] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [CANDIDATE]: Term 1 election: Requested vote from peers 11c2bc2da8be4e3aad13f70b92b75f83 (127.14.200.2:39285), 02c13bb02a3342dfb5d71af0fc000136 (127.14.200.1:41747)
I20260810 21:03:35.423583 4354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fec8daeec44df58a98ea745cdc8547" candidate_uuid: "a15373ca1a9d4a3499c075558900f6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c13bb02a3342dfb5d71af0fc000136"
I20260810 21:03:35.423585 4499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21fec8daeec44df58a98ea745cdc8547" candidate_uuid: "a15373ca1a9d4a3499c075558900f6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c2bc2da8be4e3aad13f70b92b75f83"
I20260810 21:03:35.423666 4354 raft_consensus.cc:3037] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:35.423686 4499 raft_consensus.cc:3037] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:35.424393 4499 raft_consensus.cc:2454] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a15373ca1a9d4a3499c075558900f6f5 in term 1.
I20260810 21:03:35.424472 4354 raft_consensus.cc:2454] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a15373ca1a9d4a3499c075558900f6f5 in term 1.
I20260810 21:03:35.424669 4574 leader_election.cc:302] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c2bc2da8be4e3aad13f70b92b75f83, a15373ca1a9d4a3499c075558900f6f5; no voters:
I20260810 21:03:35.424865 4720 raft_consensus.cc:2781] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:35.425002 4720 raft_consensus.cc:686] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 1 LEADER]: Becoming Leader. State: Replica: a15373ca1a9d4a3499c075558900f6f5, State: Running, Role: LEADER
I20260810 21:03:35.425096 4720 consensus_queue.cc:237] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } }
I20260810 21:03:35.426007 4196 catalog_manager.cc:5495] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a15373ca1a9d4a3499c075558900f6f5 (127.14.200.3). New cstate: current_term: 1 leader_uuid: "a15373ca1a9d4a3499c075558900f6f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a15373ca1a9d4a3499c075558900f6f5" member_type: VOTER last_known_addr { host: "127.14.200.3" port: 40251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 } health_report { overall_health: UNKNOWN } } }
I20260810 21:03:35.435945 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.433972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.14.200.3:40251)
0810 21:03:35.434051 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:35.434052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:35.434061 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:03:35.434088 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:35.434089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:35.434102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:35.434161 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:03:35.434274 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.434687 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.435333 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:35.435497 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:35.435545 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:35.435676 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:03:35.435760 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:03:35.435816 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:35.435850 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 21:03:35.435865 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 21:03:35.437747 4499 raft_consensus.cc:1261] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 1 FOLLOWER]: Refusing update from remote peer a15373ca1a9d4a3499c075558900f6f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:35.437763 4354 raft_consensus.cc:1261] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 1 FOLLOWER]: Refusing update from remote peer a15373ca1a9d4a3499c075558900f6f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:03:35.438060 4720 consensus_queue.cc:1035] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c2bc2da8be4e3aad13f70b92b75f83" member_type: VOTER last_known_addr { host: "127.14.200.2" port: 39285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:35.438215 4723 consensus_queue.cc:1035] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c13bb02a3342dfb5d71af0fc000136" member_type: VOTER last_known_addr { host: "127.14.200.1" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:03:35.440583 4726 mvcc.cc:205] Tried to move back new op lower bound from 7317077260030574592 to 7317077259982045184. Current Snapshot: MvccSnapshot[applied={T|T < 7317077260030574592}]
I20260810 21:03:35.441452 4727 mvcc.cc:205] Tried to move back new op lower bound from 7317077260030574592 to 7317077259982045184. Current Snapshot: MvccSnapshot[applied={T|T < 7317077260030574592}]
I20260810 21:03:35.443008 4197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:35.443114 4197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 21:03:35.443841 4197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0940ce7f8c7e46149fad3278aa84cc5a: Sending DeleteTablet for 3 replicas of tablet 21fec8daeec44df58a98ea745cdc8547
I20260810 21:03:35.444207 4478 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fec8daeec44df58a98ea745cdc8547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:35 UTC) from {username='slave'} at 127.0.0.1:38400
I20260810 21:03:35.444205 4624 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fec8daeec44df58a98ea745cdc8547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:35 UTC) from {username='slave'} at 127.0.0.1:46576
I20260810 21:03:35.444249 4333 tablet_service.cc:1514] Processing DeleteTablet for tablet 21fec8daeec44df58a98ea745cdc8547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 21:03:35 UTC) from {username='slave'} at 127.0.0.1:40848
I20260810 21:03:35.444453 4739 tablet_replica.cc:330] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136: stopping tablet replica
I20260810 21:03:35.444470 4738 tablet_replica.cc:330] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: stopping tablet replica
I20260810 21:03:35.444551 4738 raft_consensus.cc:2229] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 21:03:35.444550 4739 raft_consensus.cc:2229] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:35.444629 4738 raft_consensus.cc:2258] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:35.444643 4739 raft_consensus.cc:2258] T 21fec8daeec44df58a98ea745cdc8547 P 02c13bb02a3342dfb5d71af0fc000136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:35.444802 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4265
I20260810 21:03:35.446826 4738 ts_tablet_manager.cc:1905] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:35.448493 4735 tablet_replica.cc:330] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: stopping tablet replica
I20260810 21:03:35.448586 4735 raft_consensus.cc:2229] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 21:03:35.448671 4735 raft_consensus.cc:2258] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 21:03:35.448861 4738 ts_tablet_manager.cc:1918] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:35.448921 4738 log.cc:1196] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-2/wal/wals/21fec8daeec44df58a98ea745cdc8547
I20260810 21:03:35.449185 4738 ts_tablet_manager.cc:1939] T 21fec8daeec44df58a98ea745cdc8547 P a15373ca1a9d4a3499c075558900f6f5: Deleting consensus metadata
I20260810 21:03:35.449551 4181 catalog_manager.cc:4841] TS a15373ca1a9d4a3499c075558900f6f5 (127.14.200.3:40251): tablet 21fec8daeec44df58a98ea745cdc8547 (table test-table [id=2c4da92b03744ea281ad2facabd43a00]) successfully deleted
I20260810 21:03:35.450268 4735 ts_tablet_manager.cc:1905] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 21:03:35.451859 4735 ts_tablet_manager.cc:1918] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 21:03:35.451933 4735 log.cc:1196] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Deleting WAL directory at /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786395574307397-15136-0/minicluster-data/ts-1/wal/wals/21fec8daeec44df58a98ea745cdc8547
I20260810 21:03:35.452171 4735 ts_tablet_manager.cc:1939] T 21fec8daeec44df58a98ea745cdc8547 P 11c2bc2da8be4e3aad13f70b92b75f83: Deleting consensus metadata
I20260810 21:03:35.452543 4183 catalog_manager.cc:4841] TS 11c2bc2da8be4e3aad13f70b92b75f83 (127.14.200.2:39285): tablet 21fec8daeec44df58a98ea745cdc8547 (table test-table [id=2c4da92b03744ea281ad2facabd43a00]) successfully deleted
W20260810 21:03:35.459380 4184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.200.1:41747 (error 108)
I20260810 21:03:35.459534 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4406
I20260810 21:03:35.461740 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:35.461579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.1:41747 (local address 127.0.0.1:40854)
0810 21:03:35.461654 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 21:03:35.461706 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 21:03:35.463393 4184 catalog_manager.cc:4584] TS 02c13bb02a3342dfb5d71af0fc000136 (127.14.200.1:41747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21fec8daeec44df58a98ea745cdc8547: Network error: Client connection negotiation failed: client connection to 127.14.200.1:41747: connect: Connection refused (error 111)
I20260810 21:03:35.466490 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4553
I20260810 21:03:35.472688 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4165
2026-08-10T21:03:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (804 ms)
[----------] 6 tests from EncryptionPolicyTest (8499 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:35 dist-test-slave-1x4g krb5kdc[4744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:35 dist-test-slave-1x4g krb5kdc[4744](info): set up 2 sockets
Aug 10 21:03:35 dist-test-slave-1x4g krb5kdc[4744](info): commencing operation
krb5kdc: starting...
W20260810 21:03:36.540022 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:36.653549 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:36.731495 4759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:36.731736 4759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:36.731772 4759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:36.732856 4759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:36.732920 4759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:36.732942 4759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:36.732961 4759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:36.732987 4759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:36.734452 4759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:36.734736 4759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:36.734962 4759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 21:03:36.739873 4759 server_base.cc:1034] running on GCE node
W20260810 21:03:36.739817 4767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:36.739830 4764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:36.739843 4765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:36.740306 4759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:36.740618 4759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:36.741775 4759 hybrid_clock.cc:648] HybridClock initialized: now 1786395816741769 us; error 38 us; skew 500 ppm
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395816, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:36.744163 4759 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:36.744686 4759 webserver.cc:460] Webserver started at http://127.0.0.1:34491/ using document root <none> and password file <none>
I20260810 21:03:36.744930 4759 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:36.744999 4759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:36.745157 4759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:36.746182 4759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "3f6daabe9215446994fe16d641931fbb"
format_stamp: "Formatted at 2026-08-10 21:03:36 on dist-test-slave-1x4g"
server_key: "f7c89409897095fbaa0ae7a9522df050"
server_key_iv: "9036f281dea43078ba11ea8e5336c505"
server_key_version: "encryptionkey@0"
I20260810 21:03:36.746551 4759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "3f6daabe9215446994fe16d641931fbb"
format_stamp: "Formatted at 2026-08-10 21:03:36 on dist-test-slave-1x4g"
server_key: "f7c89409897095fbaa0ae7a9522df050"
server_key_iv: "9036f281dea43078ba11ea8e5336c505"
server_key_version: "encryptionkey@0"
I20260810 21:03:36.747854 4759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:36.748623 4774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:36.748811 4759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:36.748912 4759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "3f6daabe9215446994fe16d641931fbb"
format_stamp: "Formatted at 2026-08-10 21:03:36 on dist-test-slave-1x4g"
server_key: "f7c89409897095fbaa0ae7a9522df050"
server_key_iv: "9036f281dea43078ba11ea8e5336c505"
server_key_version: "encryptionkey@0"
I20260810 21:03:36.748991 4759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:36.764168 4759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:36.766407 4759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:36.766563 4759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:36.770872 4759 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40739
I20260810 21:03:36.770903 4829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40739 every 8 connection(s)
I20260810 21:03:36.771250 4759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:36.772107 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:36.774356 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb: Bootstrap starting.
I20260810 21:03:36.774870 4830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:36.775120 4830 log.cc:824] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:36.775727 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb: No bootstrap required, opened a new log
I20260810 21:03:36.777333 4830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } }
I20260810 21:03:36.777477 4830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:36.777501 4830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6daabe9215446994fe16d641931fbb, State: Initialized, Role: FOLLOWER
I20260810 21:03:36.777596 4830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } }
I20260810 21:03:36.777647 4830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:36.777667 4830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:36.777740 4830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:36.777974 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4759
I20260810 21:03:36.778074 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:36.778291 15136 external_mini_cluster.cc:1428] Setting key dde2be23a35abfd18020cd837807da7a
I20260810 21:03:36.778393 4830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } }
I20260810 21:03:36.778517 4830 leader_election.cc:302] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f6daabe9215446994fe16d641931fbb; no voters:
I20260810 21:03:36.778681 4830 leader_election.cc:288] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:36.778854 4835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:36.778941 4830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:36.778992 4835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [term 1 LEADER]: Becoming Leader. State: Replica: 3f6daabe9215446994fe16d641931fbb, State: Running, Role: LEADER
I20260810 21:03:36.779071 4835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } }
I20260810 21:03:36.779386 4835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f6daabe9215446994fe16d641931fbb. Latest consensus state: current_term: 1 leader_uuid: "3f6daabe9215446994fe16d641931fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } } }
I20260810 21:03:36.779474 4835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:36.779632 4833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f6daabe9215446994fe16d641931fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6daabe9215446994fe16d641931fbb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40739 } } }
I20260810 21:03:36.779772 4833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:36.779690 4843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:36.780393 4843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:36.782198 4843 catalog_manager.cc:1347] Generated new cluster ID: bb6c4484bd1542d281f68776b7629cbf
I20260810 21:03:36.782259 4843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 21:03:36.790647 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:36.778874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54533 (local address 127.0.0.1:40739)
0810 21:03:36.779271 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:36.779278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:36.779304 (+ 26us) server_negotiation.cc:407] Connection header received
0810 21:03:36.779450 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:36.779457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:36.779510 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:36.779651 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:36.780601 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:36.781872 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:36.782923 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:36.783539 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:36.783633 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:36.786317 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:36.786321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:36.786322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:36.786334 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:36.788857 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:36.789305 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:36.789309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:36.789309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:36.789441 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:36.789684 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:36.789685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:36.789685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:36.790024 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:36.790195 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:36.790305 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 21:03:36.790406 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":81,"threads_started":1}
I20260810 21:03:36.791630 4843 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:36.791841 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:36.792284 4843 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:36.795734 4843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f6daabe9215446994fe16d641931fbb: Generated new TSK 0
I20260810 21:03:36.795905 4843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:36.801913 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:36.793308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44937 (local address 127.0.0.1:40739)
0810 21:03:36.793338 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:36.793340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:36.795499 (+ 2159us) server_negotiation.cc:407] Connection header received
0810 21:03:36.795567 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:36.795567 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:36.795577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:36.795633 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:36.796337 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:36.796876 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:36.797881 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:36.798087 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:36.798136 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:36.799426 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:36.799427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:36.799427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:36.799433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:36.800827 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:36.801213 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:36.801214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:36.801214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:36.801263 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:36.801489 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:36.801490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:36.801490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:36.801579 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:36.801663 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:36.801812 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 21:03:36.801861 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 21:03:36.803723 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4759
W20260810 21:03:36.813102 4741 connection.cc:535] client connection to 127.0.0.1:40739 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40739): Connection reset by peer (error 104)
2026-08-10T21:03:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4865](info): set up 2 sockets
Aug 10 21:03:36 dist-test-slave-1x4g krb5kdc[4865](info): commencing operation
krb5kdc: starting...
W20260810 21:03:37.869572 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:37 dist-test-slave-1x4g krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:37.985478 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34139
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:38.066068 4878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:38.066285 4878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:38.066322 4878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:38.067441 4878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:38.067507 4878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:38.067533 4878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:38.067564 4878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:38.067588 4878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:38.069092 4878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34139
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:38.069356 4878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:38.069574 4878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:38.074223 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:38.074263 4886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:38.074302 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:38.074465 4878 server_base.cc:1034] running on GCE node
I20260810 21:03:38.074657 4878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:38.074927 4878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:38.076067 4878 hybrid_clock.cc:648] HybridClock initialized: now 1786395818076042 us; error 40 us; skew 500 ppm
Aug 10 21:03:38 dist-test-slave-1x4g krb5kdc[4865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:38.083348 4878 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:38.083770 4878 webserver.cc:460] Webserver started at http://127.0.0.1:46099/ using document root <none> and password file <none>
I20260810 21:03:38.084035 4878 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:38.084105 4878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:38.084251 4878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:38.085361 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "426aa858e6f04980834fab69ac47908b"
format_stamp: "Formatted at 2026-08-10 21:03:38 on dist-test-slave-1x4g"
server_key: "3bb70fb674fa3eb42484573a7576260d"
server_key_iv: "c3c19e32f337c3a5a4bf7b766c88fb96"
server_key_version: "encryptionkey@0"
I20260810 21:03:38.085724 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "426aa858e6f04980834fab69ac47908b"
format_stamp: "Formatted at 2026-08-10 21:03:38 on dist-test-slave-1x4g"
server_key: "3bb70fb674fa3eb42484573a7576260d"
server_key_iv: "c3c19e32f337c3a5a4bf7b766c88fb96"
server_key_version: "encryptionkey@0"
I20260810 21:03:38.087003 4878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:38.087700 4893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:38.087862 4878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:38.087957 4878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "426aa858e6f04980834fab69ac47908b"
format_stamp: "Formatted at 2026-08-10 21:03:38 on dist-test-slave-1x4g"
server_key: "3bb70fb674fa3eb42484573a7576260d"
server_key_iv: "c3c19e32f337c3a5a4bf7b766c88fb96"
server_key_version: "encryptionkey@0"
I20260810 21:03:38.088037 4878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:38.104938 4878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:38.107201 4878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:38.107360 4878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:38.111303 4878 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34139
I20260810 21:03:38.111371 4952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34139 every 8 connection(s)
I20260810 21:03:38.111747 4878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:38.112583 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:38.115100 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b: Bootstrap starting.
I20260810 21:03:38.115712 4953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:38.116024 4953 log.cc:824] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:38.116726 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b: No bootstrap required, opened a new log
I20260810 21:03:38.118374 4953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } }
I20260810 21:03:38.118513 4953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:38.118535 4953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 426aa858e6f04980834fab69ac47908b, State: Initialized, Role: FOLLOWER
I20260810 21:03:38.118633 4953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } }
I20260810 21:03:38.118711 4953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:38.118744 4953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:38.118801 4953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:38.119334 4953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } }
I20260810 21:03:38.119457 4953 leader_election.cc:302] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 426aa858e6f04980834fab69ac47908b; no voters:
I20260810 21:03:38.119629 4953 leader_election.cc:288] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:38.119673 4956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:38.119839 4956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [term 1 LEADER]: Becoming Leader. State: Replica: 426aa858e6f04980834fab69ac47908b, State: Running, Role: LEADER
I20260810 21:03:38.119920 4953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:38.119989 4956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } }
I20260810 21:03:38.120436 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 4878
I20260810 21:03:38.120527 4959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 426aa858e6f04980834fab69ac47908b. Latest consensus state: current_term: 1 leader_uuid: "426aa858e6f04980834fab69ac47908b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } } }
I20260810 21:03:38.120599 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:38.120628 4959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:38.120849 15136 external_mini_cluster.cc:1428] Setting key 119d259c5ed0149e0eae7d105f5c0c27
I20260810 21:03:38.121047 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "426aa858e6f04980834fab69ac47908b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "426aa858e6f04980834fab69ac47908b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34139 } } }
I20260810 21:03:38.121138 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:38.121296 4967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:38.121896 4967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:38.123435 4967 catalog_manager.cc:1347] Generated new cluster ID: eb3b1090256942c69e43283dce82e36a
I20260810 21:03:38.123484 4967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:38.126924 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:38.121530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37321 (local address 127.0.0.1:34139)
0810 21:03:38.121687 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:38.121695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:38.121823 (+ 128us) server_negotiation.cc:407] Connection header received
0810 21:03:38.121964 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:38.121972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:38.122030 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:38.122175 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:38.122405 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:38.123598 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:38.124733 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:38.125376 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:38.125475 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:38.126180 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:38.126184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:38.126184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:38.126379 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:38.126416 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:38.126428 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:38.126504 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:38.126615 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 21:03:38.126725 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260810 21:03:38.127831 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:38.131106 4967 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:38.131774 4967 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:38.133064 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:38.129459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50457 (local address 127.0.0.1:34139)
0810 21:03:38.129505 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:38.129507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:38.129730 (+ 223us) server_negotiation.cc:407] Connection header received
0810 21:03:38.129774 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:38.129774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:38.129785 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:38.129834 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:38.129983 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:38.130718 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:38.131669 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:38.131978 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:38.132059 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:38.132806 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:38.132807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:38.132807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:38.132812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:38.132828 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:38.132832 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:38.132909 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:38.132979 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 21:03:38.132991 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:03:38.135723 4967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 426aa858e6f04980834fab69ac47908b: Generated new TSK 0
I20260810 21:03:38.135900 4967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:38.139425 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 4878
W20260810 21:03:38.151373 4863 connection.cc:535] client connection to 127.0.0.1:34139 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34139): Connection reset by peer (error 104)
2026-08-10T21:03:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:38 dist-test-slave-1x4g krb5kdc[4986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:38 dist-test-slave-1x4g krb5kdc[4986](info): set up 2 sockets
Aug 10 21:03:38 dist-test-slave-1x4g krb5kdc[4986](info): commencing operation
krb5kdc: starting...
W20260810 21:03:39.205112 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:39 dist-test-slave-1x4g krb5kdc[4986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:39.320855 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:39.401342 5002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:39.401576 5002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:39.401612 5002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:39.402689 5002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:39.402757 5002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:39.402781 5002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:39.402809 5002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:39.402839 5002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:39.404260 5002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:39.404526 5002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:39.404739 5002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:39.409376 5008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:39.409487 5002 server_base.cc:1034] running on GCE node
W20260810 21:03:39.409379 5010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:39.409411 5007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:39.409857 5002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:39.410132 5002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:39.411293 5002 hybrid_clock.cc:648] HybridClock initialized: now 1786395819411279 us; error 43 us; skew 500 ppm
Aug 10 21:03:39 dist-test-slave-1x4g krb5kdc[4986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:39.413686 5002 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:39.414090 5002 webserver.cc:460] Webserver started at http://127.0.0.1:36065/ using document root <none> and password file <none>
I20260810 21:03:39.414309 5002 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:39.414371 5002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:39.414513 5002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:39.415494 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "d43cd1bbebc14dba81aaeb50bfac04ae"
format_stamp: "Formatted at 2026-08-10 21:03:39 on dist-test-slave-1x4g"
server_key: "c4da398368907d231054510ef229a868"
server_key_iv: "191dd5081df6428ce7f37c83c4db5a2a"
server_key_version: "encryptionkey@0"
I20260810 21:03:39.415876 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "d43cd1bbebc14dba81aaeb50bfac04ae"
format_stamp: "Formatted at 2026-08-10 21:03:39 on dist-test-slave-1x4g"
server_key: "c4da398368907d231054510ef229a868"
server_key_iv: "191dd5081df6428ce7f37c83c4db5a2a"
server_key_version: "encryptionkey@0"
I20260810 21:03:39.417285 5002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:39.418025 5017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:39.418183 5002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:39.418285 5002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "d43cd1bbebc14dba81aaeb50bfac04ae"
format_stamp: "Formatted at 2026-08-10 21:03:39 on dist-test-slave-1x4g"
server_key: "c4da398368907d231054510ef229a868"
server_key_iv: "191dd5081df6428ce7f37c83c4db5a2a"
server_key_version: "encryptionkey@0"
I20260810 21:03:39.418366 5002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:39.421689 5002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:39.423413 5002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:39.423570 5002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:39.427543 5002 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33551
I20260810 21:03:39.427587 5076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33551 every 8 connection(s)
I20260810 21:03:39.427891 5002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:39.428745 5077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:39.431068 5077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae: Bootstrap starting.
I20260810 21:03:39.431650 5077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:39.431948 5077 log.cc:824] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:39.432646 5077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae: No bootstrap required, opened a new log
I20260810 21:03:39.434243 5077 raft_consensus.cc:348] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260810 21:03:39.434393 5077 raft_consensus.cc:374] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:39.434464 5077 raft_consensus.cc:729] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d43cd1bbebc14dba81aaeb50bfac04ae, State: Initialized, Role: FOLLOWER
I20260810 21:03:39.434625 5077 consensus_queue.cc:260] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260810 21:03:39.434706 5077 raft_consensus.cc:388] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:39.434751 5077 raft_consensus.cc:482] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:39.434795 5077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:39.435288 5077 raft_consensus.cc:504] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260810 21:03:39.435403 5077 leader_election.cc:302] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d43cd1bbebc14dba81aaeb50bfac04ae; no voters:
I20260810 21:03:39.435587 5077 leader_election.cc:288] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:39.435632 5080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:39.435875 5077 sys_catalog.cc:564] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:39.435878 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5002
I20260810 21:03:39.435956 5080 raft_consensus.cc:686] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [term 1 LEADER]: Becoming Leader. State: Replica: d43cd1bbebc14dba81aaeb50bfac04ae, State: Running, Role: LEADER
I20260810 21:03:39.435997 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:39.436082 5080 consensus_queue.cc:237] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } }
I20260810 21:03:39.436200 15136 external_mini_cluster.cc:1428] Setting key eef013a942ba57093a7e7b24d8038242
I20260810 21:03:39.436460 5080 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader d43cd1bbebc14dba81aaeb50bfac04ae. Latest consensus state: current_term: 1 leader_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } } }
I20260810 21:03:39.436569 5080 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:39.436753 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cd1bbebc14dba81aaeb50bfac04ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33551 } } }
I20260810 21:03:39.436873 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:39.437414 5090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:39.438618 5090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:39.439021 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:39.436897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37523 (local address 127.0.0.1:33551)
0810 21:03:39.437082 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:39.437092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:39.437193 (+ 101us) server_negotiation.cc:407] Connection header received
0810 21:03:39.437362 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:39.437369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:39.437463 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:39.437609 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:39.438389 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:39.438393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:39.438393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:39.438397 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:39.438433 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:39.438447 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:39.438507 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:39.438803 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 21:03:39.438845 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":110,"threads_started":1}
I20260810 21:03:39.439911 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:39.440562 5090 catalog_manager.cc:1347] Generated new cluster ID: aa3d5b35033f44f5a7b68e2d799e30a9
I20260810 21:03:39.440614 5090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:39.442626 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:39.441509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51569 (local address 127.0.0.1:33551)
0810 21:03:39.441546 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:39.441547 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:39.441670 (+ 123us) server_negotiation.cc:407] Connection header received
0810 21:03:39.441753 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:39.441754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:39.441759 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:39.441807 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:39.442409 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:39.442410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:39.442410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:39.442414 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:39.442427 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:39.442431 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:39.442476 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:39.442535 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 21:03:39.442549 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:03:39.447136 5090 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:39.447309 5090 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:39.454075 5090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d43cd1bbebc14dba81aaeb50bfac04ae: Generated new TSK 0
I20260810 21:03:39.454242 5090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:39.460888 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5002
2026-08-10T21:03:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:39 dist-test-slave-1x4g krb5kdc[5111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:39 dist-test-slave-1x4g krb5kdc[5111](info): set up 2 sockets
Aug 10 21:03:39 dist-test-slave-1x4g krb5kdc[5111](info): commencing operation
krb5kdc: starting...
W20260810 21:03:40.513487 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:40.629644 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:40.711599 5125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:40.711889 5125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:40.711939 5125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:40.713104 5125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:40.713162 5125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:40.713186 5125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:40.713219 5125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:40.713241 5125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:40.714699 5125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:40.714965 5125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:40.715175 5125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:40.720005 5130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:40.720124 5131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:40.720199 5125 server_base.cc:1034] running on GCE node
W20260810 21:03:40.720016 5133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:40.720583 5125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:40.720911 5125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:40.722086 5125 hybrid_clock.cc:648] HybridClock initialized: now 1786395820722074 us; error 36 us; skew 500 ppm
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395820, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:40.724505 5125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:40.724977 5125 webserver.cc:460] Webserver started at http://127.0.0.1:43493/ using document root <none> and password file <none>
I20260810 21:03:40.725229 5125 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:40.725291 5125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:40.725416 5125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:40.726395 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "4b4213b9e7c3457faa78cff7bed99fd7"
format_stamp: "Formatted at 2026-08-10 21:03:40 on dist-test-slave-1x4g"
server_key: "111197990ceebf63f60c1fb1585f8ba8"
server_key_iv: "cda32bb6fd3e15a56de79c2613a1e46c"
server_key_version: "encryptionkey@0"
I20260810 21:03:40.726754 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "4b4213b9e7c3457faa78cff7bed99fd7"
format_stamp: "Formatted at 2026-08-10 21:03:40 on dist-test-slave-1x4g"
server_key: "111197990ceebf63f60c1fb1585f8ba8"
server_key_iv: "cda32bb6fd3e15a56de79c2613a1e46c"
server_key_version: "encryptionkey@0"
I20260810 21:03:40.728049 5125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:03:40.728686 5141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:40.728855 5125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:40.728955 5125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "4b4213b9e7c3457faa78cff7bed99fd7"
format_stamp: "Formatted at 2026-08-10 21:03:40 on dist-test-slave-1x4g"
server_key: "111197990ceebf63f60c1fb1585f8ba8"
server_key_iv: "cda32bb6fd3e15a56de79c2613a1e46c"
server_key_version: "encryptionkey@0"
I20260810 21:03:40.729035 5125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:40.742683 5125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:40.744720 5125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:40.744882 5125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:40.748811 5125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46299
I20260810 21:03:40.748865 5199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46299 every 8 connection(s)
I20260810 21:03:40.749233 5125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:40.750049 5200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:40.752250 5200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: Bootstrap starting.
I20260810 21:03:40.752885 5200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:40.753166 5200 log.cc:824] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:40.753813 5200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: No bootstrap required, opened a new log
I20260810 21:03:40.754493 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5125
I20260810 21:03:40.754618 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:40.754807 15136 external_mini_cluster.cc:1428] Setting key 3b3bbdb326c49549dc26359b7275a182
I20260810 21:03:40.756091 5200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } }
I20260810 21:03:40.756219 5200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:40.756239 5200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4213b9e7c3457faa78cff7bed99fd7, State: Initialized, Role: FOLLOWER
I20260810 21:03:40.756356 5200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } }
I20260810 21:03:40.756424 5200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:40.756448 5200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:40.756541 5200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:40.757202 5200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } }
I20260810 21:03:40.757319 5200 leader_election.cc:302] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b4213b9e7c3457faa78cff7bed99fd7; no voters:
I20260810 21:03:40.757501 5200 leader_election.cc:288] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:40.757534 5205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:40.757642 5205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b4213b9e7c3457faa78cff7bed99fd7, State: Running, Role: LEADER
I20260810 21:03:40.757751 5200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:40.757771 5205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } }
I20260810 21:03:40.758129 5205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b4213b9e7c3457faa78cff7bed99fd7. Latest consensus state: current_term: 1 leader_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } } }
I20260810 21:03:40.758144 5207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4213b9e7c3457faa78cff7bed99fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46299 } } }
I20260810 21:03:40.758263 5205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:40.758339 5207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7 [sys.catalog]: This master's current role is: LEADER
W20260810 21:03:40.759559 5221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:03:40.759902 5221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:03:40.759986 5214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:40.760161 5214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:40.761605 5214 catalog_manager.cc:1347] Generated new cluster ID: 08fed882d3fa42568fed14a4efe08f7a
I20260810 21:03:40.761656 5214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 21:03:40.766700 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:40.755290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45391 (local address 127.0.0.1:46299)
0810 21:03:40.755453 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:40.755461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:40.755483 (+ 22us) server_negotiation.cc:407] Connection header received
0810 21:03:40.756293 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:40.756309 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:40.756389 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:40.756721 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:40.757447 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:40.758742 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:40.759663 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:40.760257 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:40.760377 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:40.763188 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:40.763191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:40.763192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:40.763202 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:40.765127 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:40.765513 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:40.765516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:40.765516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:40.765611 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:40.765822 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:40.765823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:40.765823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:40.766127 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:40.766278 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:40.766407 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 21:03:40.766525 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260810 21:03:40.767493 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:40.775770 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:40.769134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58125 (local address 127.0.0.1:46299)
0810 21:03:40.769160 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:40.769161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:40.769168 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:03:40.769229 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:40.769229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:40.769237 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:40.769285 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:40.769881 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:40.770544 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:40.771573 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:40.771863 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:40.771919 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:40.773279 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:40.773281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:40.773281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:40.773287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:40.774687 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:40.775038 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:40.775039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:40.775040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:40.775090 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:40.775280 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:40.775281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:40.775281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:40.775390 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:40.775489 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:40.775630 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 21:03:40.775698 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260810 21:03:40.776705 5214 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:40.777209 5214 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:40.786787 5214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b4213b9e7c3457faa78cff7bed99fd7: Generated new TSK 0
I20260810 21:03:40.786980 5214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:40.797181 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5125
2026-08-10T21:03:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5232](info): set up 2 sockets
Aug 10 21:03:40 dist-test-slave-1x4g krb5kdc[5232](info): commencing operation
krb5kdc: starting...
W20260810 21:03:41.876725 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:41 dist-test-slave-1x4g krb5kdc[5232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:41.991295 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:42.070737 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:42.071002 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:42.071041 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:42.072201 5246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:42.072269 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:42.072294 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:42.072324 5246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:42.072356 5246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:42.073810 5246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:42.074079 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:42.074296 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:42.079135 5255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:42.079331 5246 server_base.cc:1034] running on GCE node
W20260810 21:03:42.079252 5253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:42.079144 5252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:42.079655 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:42.079994 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:42.081164 5246 hybrid_clock.cc:648] HybridClock initialized: now 1786395822081145 us; error 30 us; skew 500 ppm
Aug 10 21:03:42 dist-test-slave-1x4g krb5kdc[5232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:42.083499 5246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:42.083905 5246 webserver.cc:460] Webserver started at http://127.0.0.1:35399/ using document root <none> and password file <none>
I20260810 21:03:42.084120 5246 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:42.084183 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:42.084328 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:42.085400 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "f14b21c9142a47baa67a40daa00c0797"
format_stamp: "Formatted at 2026-08-10 21:03:42 on dist-test-slave-1x4g"
server_key: "72bca851ef38ff11def07458eea9b093"
server_key_iv: "cb4b9525d9bdc78a8fd577154283da1c"
server_key_version: "encryptionkey@0"
I20260810 21:03:42.085778 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "f14b21c9142a47baa67a40daa00c0797"
format_stamp: "Formatted at 2026-08-10 21:03:42 on dist-test-slave-1x4g"
server_key: "72bca851ef38ff11def07458eea9b093"
server_key_iv: "cb4b9525d9bdc78a8fd577154283da1c"
server_key_version: "encryptionkey@0"
I20260810 21:03:42.087042 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:42.087708 5262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:42.087859 5246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:42.087957 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "f14b21c9142a47baa67a40daa00c0797"
format_stamp: "Formatted at 2026-08-10 21:03:42 on dist-test-slave-1x4g"
server_key: "72bca851ef38ff11def07458eea9b093"
server_key_iv: "cb4b9525d9bdc78a8fd577154283da1c"
server_key_version: "encryptionkey@0"
I20260810 21:03:42.088030 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:42.103580 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:42.105408 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:42.105580 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:42.109731 5246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40575
I20260810 21:03:42.109779 5319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40575 every 8 connection(s)
I20260810 21:03:42.110127 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:42.111066 5321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:42.113451 5321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797: Bootstrap starting.
I20260810 21:03:42.114003 5321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:42.114282 5321 log.cc:824] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:42.114990 5321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797: No bootstrap required, opened a new log
I20260810 21:03:42.116179 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5246
I20260810 21:03:42.116282 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:42.116499 15136 external_mini_cluster.cc:1428] Setting key 5896827bc512d53bf4da5e72c4839ab9
I20260810 21:03:42.116762 5321 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260810 21:03:42.116999 5321 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:42.117048 5321 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14b21c9142a47baa67a40daa00c0797, State: Initialized, Role: FOLLOWER
I20260810 21:03:42.117162 5321 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260810 21:03:42.117252 5321 raft_consensus.cc:388] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:42.117285 5321 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:42.117331 5321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:42.118084 5321 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260810 21:03:42.118191 5321 leader_election.cc:302] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f14b21c9142a47baa67a40daa00c0797; no voters:
I20260810 21:03:42.118333 5321 leader_election.cc:288] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:42.118405 5327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:42.118539 5327 raft_consensus.cc:686] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [term 1 LEADER]: Becoming Leader. State: Replica: f14b21c9142a47baa67a40daa00c0797, State: Running, Role: LEADER
I20260810 21:03:42.118574 5321 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:42.118626 5327 consensus_queue.cc:237] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260810 21:03:42.118963 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f14b21c9142a47baa67a40daa00c0797. Latest consensus state: current_term: 1 leader_uuid: "f14b21c9142a47baa67a40daa00c0797" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } } }
I20260810 21:03:42.119062 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:42.118971 5328 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f14b21c9142a47baa67a40daa00c0797" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b21c9142a47baa67a40daa00c0797" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } } }
I20260810 21:03:42.119335 5328 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:42.119438 5335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:42.119920 5335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:42.121796 5335 catalog_manager.cc:1347] Generated new cluster ID: 80c13cdc3cd44826be500071d0471a2b
I20260810 21:03:42.121855 5335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:42.122622 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:42.117008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38519 (local address 127.0.0.1:40575)
0810 21:03:42.117195 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:42.117204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:42.117237 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:03:42.117363 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:42.117370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:42.117426 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:42.117578 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:42.118054 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:42.119405 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:42.120609 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:42.121098 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:42.121218 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:42.121921 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:42.121925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:42.121925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:42.122123 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:42.122162 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:42.122176 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:42.122248 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:42.122339 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 21:03:42.122402 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260810 21:03:42.123520 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:42.140177 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:42.125073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59603 (local address 127.0.0.1:40575)
0810 21:03:42.125103 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:42.125105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:42.136938 (+ 11833us) server_negotiation.cc:407] Connection header received
0810 21:03:42.136981 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:42.136982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:42.136992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:42.137047 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:42.137190 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:42.137879 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:42.138897 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:42.139202 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:42.139259 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:42.139932 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:42.139933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:42.139934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:42.139942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:42.139956 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:42.139960 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:42.140007 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:42.140083 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 21:03:42.140107 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 21:03:42.162147 5335 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:42.162828 5335 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:42.166769 5335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f14b21c9142a47baa67a40daa00c0797: Generated new TSK 0
I20260810 21:03:42.166954 5335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:42.173650 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5246
2026-08-10T21:03:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:42 dist-test-slave-1x4g krb5kdc[5356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:42 dist-test-slave-1x4g krb5kdc[5356](info): set up 2 sockets
Aug 10 21:03:42 dist-test-slave-1x4g krb5kdc[5356](info): commencing operation
krb5kdc: starting...
W20260810 21:03:43.226195 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:43 dist-test-slave-1x4g krb5kdc[5356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:43.342978 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:43.421507 5371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:43.421751 5371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:43.421788 5371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:43.422924 5371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:43.422991 5371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:43.423014 5371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:43.423034 5371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:43.423055 5371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:43.424718 5371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:43.425089 5371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:43.425350 5371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:43.430310 5377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:43.430430 5371 server_base.cc:1034] running on GCE node
W20260810 21:03:43.430302 5379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:43.430305 5376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:43.430748 5371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:43.431057 5371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:43.432204 5371 hybrid_clock.cc:648] HybridClock initialized: now 1786395823432188 us; error 26 us; skew 500 ppm
Aug 10 21:03:43 dist-test-slave-1x4g krb5kdc[5356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:43.439317 5371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:43.439765 5371 webserver.cc:460] Webserver started at http://127.0.0.1:37699/ using document root <none> and password file <none>
I20260810 21:03:43.439994 5371 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:43.440063 5371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:43.440214 5371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:43.441318 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "4b1a2c20464b4f39a22290266022a98a"
format_stamp: "Formatted at 2026-08-10 21:03:43 on dist-test-slave-1x4g"
server_key: "d3ec252a92fef5536d1ab32ba900ab8a"
server_key_iv: "bdda17e468c19f1ecabc49db273d2272"
server_key_version: "encryptionkey@0"
I20260810 21:03:43.441699 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "4b1a2c20464b4f39a22290266022a98a"
format_stamp: "Formatted at 2026-08-10 21:03:43 on dist-test-slave-1x4g"
server_key: "d3ec252a92fef5536d1ab32ba900ab8a"
server_key_iv: "bdda17e468c19f1ecabc49db273d2272"
server_key_version: "encryptionkey@0"
I20260810 21:03:43.442965 5371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 21:03:43.443635 5386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:43.443794 5371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:43.443897 5371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "4b1a2c20464b4f39a22290266022a98a"
format_stamp: "Formatted at 2026-08-10 21:03:43 on dist-test-slave-1x4g"
server_key: "d3ec252a92fef5536d1ab32ba900ab8a"
server_key_iv: "bdda17e468c19f1ecabc49db273d2272"
server_key_version: "encryptionkey@0"
I20260810 21:03:43.443966 5371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:43.447005 5371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:43.448745 5371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:43.448935 5371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:43.452543 5371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45873
I20260810 21:03:43.452608 5444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45873 every 8 connection(s)
I20260810 21:03:43.452960 5371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:43.453755 5445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:43.455866 5445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a: Bootstrap starting.
I20260810 21:03:43.456439 5445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:43.456734 5445 log.cc:824] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:43.457408 5445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a: No bootstrap required, opened a new log
I20260810 21:03:43.457626 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5371
I20260810 21:03:43.457751 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:43.457942 15136 external_mini_cluster.cc:1428] Setting key f9c60f00b8d4df7947309901832a81a0
I20260810 21:03:43.459805 5445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } }
I20260810 21:03:43.459966 5445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:43.460012 5445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1a2c20464b4f39a22290266022a98a, State: Initialized, Role: FOLLOWER
I20260810 21:03:43.460143 5445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } }
I20260810 21:03:43.460244 5445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:43.460295 5445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:43.460348 5445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:43.460743 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:43.458417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43747 (local address 127.0.0.1:45873)
0810 21:03:43.458575 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:43.458582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:43.458625 (+ 43us) server_negotiation.cc:407] Connection header received
0810 21:03:43.459333 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:43.459342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:43.459374 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:43.459535 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:43.460119 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:43.460122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:43.460123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:43.460127 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:43.460169 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:43.460182 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:43.460256 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:43.460460 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 21:03:43.460513 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260810 21:03:43.461308 5445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } }
I20260810 21:03:43.461426 5445 leader_election.cc:302] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1a2c20464b4f39a22290266022a98a; no voters:
I20260810 21:03:43.461591 5445 leader_election.cc:288] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:43.461637 5450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:43.461777 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:43.461793 5450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [term 1 LEADER]: Becoming Leader. State: Replica: 4b1a2c20464b4f39a22290266022a98a, State: Running, Role: LEADER
I20260810 21:03:43.461853 5445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:43.461899 5450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } }
I20260810 21:03:43.462296 5453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b1a2c20464b4f39a22290266022a98a. Latest consensus state: current_term: 1 leader_uuid: "4b1a2c20464b4f39a22290266022a98a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } } }
I20260810 21:03:43.462332 5452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b1a2c20464b4f39a22290266022a98a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1a2c20464b4f39a22290266022a98a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45873 } } }
I20260810 21:03:43.462659 5452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:43.462643 5453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:43.462989 5460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:43.463490 5460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:43.465111 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:43.463905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47509 (local address 127.0.0.1:45873)
0810 21:03:43.463938 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:43.463941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:43.464168 (+ 227us) server_negotiation.cc:407] Connection header received
0810 21:03:43.464242 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:43.464243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:43.464248 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:43.464303 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:43.464850 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:43.464851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:43.464851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:43.464855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:43.464868 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:43.464871 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:43.464925 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:43.465044 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 21:03:43.465055 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 21:03:43.465317 5460 catalog_manager.cc:1347] Generated new cluster ID: 1e0283dd253143aba81d56637fe198f7
I20260810 21:03:43.465360 5460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:43.473325 5460 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:43.473474 5460 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:43.482999 5460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b1a2c20464b4f39a22290266022a98a: Generated new TSK 0
I20260810 21:03:43.483157 5460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:43.488219 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5371
2026-08-10T21:03:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:43 dist-test-slave-1x4g krb5kdc[5478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:43 dist-test-slave-1x4g krb5kdc[5478](info): set up 2 sockets
Aug 10 21:03:43 dist-test-slave-1x4g krb5kdc[5478](info): commencing operation
krb5kdc: starting...
W20260810 21:03:44.538869 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:44.654953 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:44.733937 5493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:44.734118 5493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:44.734138 5493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:44.735206 5493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:44.735242 5493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:44.735255 5493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:44.735268 5493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:44.735280 5493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:44.736656 5493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:44.736896 5493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:44.737072 5493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:44.741597 5499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:44.741609 5502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:44.741605 5500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:44.741914 5493 server_base.cc:1034] running on GCE node
I20260810 21:03:44.742178 5493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:44.742457 5493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:44.743628 5493 hybrid_clock.cc:648] HybridClock initialized: now 1786395824743629 us; error 42 us; skew 500 ppm
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:44.746057 5493 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:44.746495 5493 webserver.cc:460] Webserver started at http://127.0.0.1:41239/ using document root <none> and password file <none>
I20260810 21:03:44.746711 5493 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:44.746775 5493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:44.746920 5493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:44.747893 5493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "52f19fdb2709460b993ebf64992127f4"
format_stamp: "Formatted at 2026-08-10 21:03:44 on dist-test-slave-1x4g"
server_key: "093a80b91920bc70dfaa2b10ae757a1c"
server_key_iv: "ac9ca3c3e2ec869a84366fa2f13c7c2e"
server_key_version: "encryptionkey@0"
I20260810 21:03:44.748226 5493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "52f19fdb2709460b993ebf64992127f4"
format_stamp: "Formatted at 2026-08-10 21:03:44 on dist-test-slave-1x4g"
server_key: "093a80b91920bc70dfaa2b10ae757a1c"
server_key_iv: "ac9ca3c3e2ec869a84366fa2f13c7c2e"
server_key_version: "encryptionkey@0"
I20260810 21:03:44.749605 5493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:44.750286 5509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:44.750450 5493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:44.750559 5493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "52f19fdb2709460b993ebf64992127f4"
format_stamp: "Formatted at 2026-08-10 21:03:44 on dist-test-slave-1x4g"
server_key: "093a80b91920bc70dfaa2b10ae757a1c"
server_key_iv: "ac9ca3c3e2ec869a84366fa2f13c7c2e"
server_key_version: "encryptionkey@0"
I20260810 21:03:44.750630 5493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:44.758484 5493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:44.760308 5493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:44.760455 5493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:44.764456 5493 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39117
I20260810 21:03:44.764484 5568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39117 every 8 connection(s)
I20260810 21:03:44.764879 5493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:44.765763 5570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:44.767915 5570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4: Bootstrap starting.
I20260810 21:03:44.768460 5570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:44.768860 5570 log.cc:824] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:44.769418 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5493
I20260810 21:03:44.769556 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:44.769636 5570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4: No bootstrap required, opened a new log
I20260810 21:03:44.769775 15136 external_mini_cluster.cc:1428] Setting key 2310aa93330a965af580013a845f5036
I20260810 21:03:44.772094 5570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260810 21:03:44.772255 5570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:44.772305 5570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f19fdb2709460b993ebf64992127f4, State: Initialized, Role: FOLLOWER
I20260810 21:03:44.772459 5570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260810 21:03:44.772552 5570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:44.772595 5570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:44.772642 5570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:44.773221 5570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260810 21:03:44.773336 5570 leader_election.cc:302] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52f19fdb2709460b993ebf64992127f4; no voters:
I20260810 21:03:44.773504 5570 leader_election.cc:288] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:44.773567 5576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:44.773741 5576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [term 1 LEADER]: Becoming Leader. State: Replica: 52f19fdb2709460b993ebf64992127f4, State: Running, Role: LEADER
I20260810 21:03:44.773845 5576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } }
I20260810 21:03:44.773741 5570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:44.774180 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52f19fdb2709460b993ebf64992127f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } } }
I20260810 21:03:44.774219 5578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f19fdb2709460b993ebf64992127f4. Latest consensus state: current_term: 1 leader_uuid: "52f19fdb2709460b993ebf64992127f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f19fdb2709460b993ebf64992127f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39117 } } }
I20260810 21:03:44.774267 5578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:44.774266 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:44.774479 5580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:44.774995 5580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:44.776577 5580 catalog_manager.cc:1347] Generated new cluster ID: ad145da5241243158105bcb72d160c8d
I20260810 21:03:44.776643 5580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 21:03:44.780985 5575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:44.770362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40655 (local address 127.0.0.1:39117)
0810 21:03:44.770531 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:44.770539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:44.770577 (+ 38us) server_negotiation.cc:407] Connection header received
0810 21:03:44.770994 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:44.771001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:44.771070 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:44.771177 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:44.771988 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:44.773307 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:44.774364 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:44.775004 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:44.775092 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:44.777824 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:44.777828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:44.777828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:44.777839 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:44.779508 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:44.779899 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:44.779902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:44.779903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:44.780005 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:44.780232 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:44.780234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:44.780234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:44.780450 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:44.780625 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:44.780708 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 21:03:44.780835 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260810 21:03:44.781978 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:44.790427 5580 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:44.790555 5575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:44.783294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40521 (local address 127.0.0.1:39117)
0810 21:03:44.783325 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:44.783326 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:44.784990 (+ 1664us) server_negotiation.cc:407] Connection header received
0810 21:03:44.785056 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:44.785057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:44.785066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:44.785115 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:44.785593 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:44.786056 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:44.786719 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:44.786921 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:44.786996 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:44.788029 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:44.788030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:44.788031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:44.788036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:44.789293 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:44.789638 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:44.789639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:44.789639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:44.789688 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:44.789912 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:44.789913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:44.789913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:44.790037 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:44.790123 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:44.790331 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 21:03:44.790388 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 21:03:44.791038 5580 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:44.799937 5580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52f19fdb2709460b993ebf64992127f4: Generated new TSK 0
I20260810 21:03:44.800112 5580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:44.802047 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5493
W20260810 21:03:44.810581 5476 connection.cc:535] client connection to 127.0.0.1:39117 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39117): Connection reset by peer (error 104)
2026-08-10T21:03:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5606](info): set up 2 sockets
Aug 10 21:03:44 dist-test-slave-1x4g krb5kdc[5606](info): commencing operation
krb5kdc: starting...
W20260810 21:03:45.859936 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:45 dist-test-slave-1x4g krb5kdc[5606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:45.972725 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:46.050908 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:46.051115 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:46.051177 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:46.052309 5621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:46.052376 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:46.052399 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:46.052428 5621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:46.052450 5621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:46.053752 5621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:46.053983 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:46.054170 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:46.058753 5627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:46.058755 5628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:46.058773 5630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:46.058770 5621 server_base.cc:1034] running on GCE node
I20260810 21:03:46.059243 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:46.059514 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:46.060678 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786395826060664 us; error 26 us; skew 500 ppm
Aug 10 21:03:46 dist-test-slave-1x4g krb5kdc[5606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:46.063117 5621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:46.063516 5621 webserver.cc:460] Webserver started at http://127.0.0.1:38003/ using document root <none> and password file <none>
I20260810 21:03:46.063725 5621 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:46.063810 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:46.063961 5621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:46.065026 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "6059fc2c97784260b16d2f88d6b29123"
format_stamp: "Formatted at 2026-08-10 21:03:46 on dist-test-slave-1x4g"
server_key: "da2c5f0002c3409efe144749b15f0a8e"
server_key_iv: "d7fdc3ccc22cb989387972825b0eb017"
server_key_version: "encryptionkey@0"
I20260810 21:03:46.065363 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "6059fc2c97784260b16d2f88d6b29123"
format_stamp: "Formatted at 2026-08-10 21:03:46 on dist-test-slave-1x4g"
server_key: "da2c5f0002c3409efe144749b15f0a8e"
server_key_iv: "d7fdc3ccc22cb989387972825b0eb017"
server_key_version: "encryptionkey@0"
I20260810 21:03:46.066591 5621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:46.067361 5639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:46.067539 5621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:46.067652 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "6059fc2c97784260b16d2f88d6b29123"
format_stamp: "Formatted at 2026-08-10 21:03:46 on dist-test-slave-1x4g"
server_key: "da2c5f0002c3409efe144749b15f0a8e"
server_key_iv: "d7fdc3ccc22cb989387972825b0eb017"
server_key_version: "encryptionkey@0"
I20260810 21:03:46.067726 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:46.079732 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:46.081905 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:46.082060 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:46.085877 5621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44335
I20260810 21:03:46.085913 5696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44335 every 8 connection(s)
I20260810 21:03:46.086225 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:46.087064 5697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:46.087080 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5621
I20260810 21:03:46.087194 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:46.087392 15136 external_mini_cluster.cc:1428] Setting key f006752a28e96ab4d43e6d639b7520a4
I20260810 21:03:46.089892 5697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123: Bootstrap starting.
I20260810 21:03:46.090456 5697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:46.090739 5697 log.cc:824] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:46.091431 5697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123: No bootstrap required, opened a new log
I20260810 21:03:46.092473 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:46.087892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:37323 (local address 127.0.0.1:44335)
0810 21:03:46.088086 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:46.088107 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:46.088135 (+ 28us) server_negotiation.cc:407] Connection header received
0810 21:03:46.088580 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:46.088587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:46.088632 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:46.088770 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:46.088989 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:46.089812 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:46.090639 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:46.091063 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:46.091197 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:46.091897 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:46.091901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:46.091901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:46.092055 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:46.092083 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:46.092093 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:46.092182 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:46.092279 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 21:03:46.092306 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260810 21:03:46.093533 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:46.093664 5697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } }
I20260810 21:03:46.093803 5697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:46.093844 5697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6059fc2c97784260b16d2f88d6b29123, State: Initialized, Role: FOLLOWER
I20260810 21:03:46.093950 5697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } }
I20260810 21:03:46.094033 5697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:46.094074 5697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:46.094117 5697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:46.094622 5697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } }
I20260810 21:03:46.094739 5697 leader_election.cc:302] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6059fc2c97784260b16d2f88d6b29123; no voters:
I20260810 21:03:46.094880 5697 leader_election.cc:288] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:46.094997 5706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:46.095100 5697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:46.095155 5706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [term 1 LEADER]: Becoming Leader. State: Replica: 6059fc2c97784260b16d2f88d6b29123, State: Running, Role: LEADER
I20260810 21:03:46.095234 5706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } }
I20260810 21:03:46.095592 5708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6059fc2c97784260b16d2f88d6b29123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } } }
I20260810 21:03:46.095754 5708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:46.095608 5710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6059fc2c97784260b16d2f88d6b29123. Latest consensus state: current_term: 1 leader_uuid: "6059fc2c97784260b16d2f88d6b29123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6059fc2c97784260b16d2f88d6b29123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44335 } } }
I20260810 21:03:46.095929 5710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:46.095996 5719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:46.096505 5719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:46.099385 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:46.094899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:54495 (local address 127.0.0.1:44335)
0810 21:03:46.094936 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:46.094938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:46.096619 (+ 1681us) server_negotiation.cc:407] Connection header received
0810 21:03:46.096743 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:46.096744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:46.096755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:46.096802 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:46.097043 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:46.097575 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:46.098278 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:46.098499 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:46.098572 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:46.099176 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:46.099177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:46.099178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:46.099182 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:46.099193 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:46.099197 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:46.099248 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:46.099305 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 21:03:46.099314 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:03:46.105110 5719 catalog_manager.cc:1347] Generated new cluster ID: f9cb882ddea54b8e945bb92dc420f72e
I20260810 21:03:46.105167 5719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:46.116582 5719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:46.117154 5719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:46.124820 5719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6059fc2c97784260b16d2f88d6b29123: Generated new TSK 0
I20260810 21:03:46.125013 5719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:46.127895 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5621
W20260810 21:03:46.135074 5604 connection.cc:535] client connection to 127.0.0.1:44335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44335): Connection reset by peer (error 104)
2026-08-10T21:03:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:46 dist-test-slave-1x4g krb5kdc[5733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:46 dist-test-slave-1x4g krb5kdc[5733](info): set up 2 sockets
Aug 10 21:03:46 dist-test-slave-1x4g krb5kdc[5733](info): commencing operation
krb5kdc: starting...
W20260810 21:03:47.178220 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:47 dist-test-slave-1x4g krb5kdc[5733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:47.289062 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:47.366753 5746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:47.366992 5746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:47.367029 5746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:47.368148 5746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:47.368214 5746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:47.368239 5746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:47.368265 5746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:47.368296 5746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:47.369746 5746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:47.370008 5746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:47.370217 5746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:47.374965 5754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:47.375131 5746 server_base.cc:1034] running on GCE node
W20260810 21:03:47.374979 5757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:47.374991 5755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:47.375461 5746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:47.375734 5746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:47.376847 5746 hybrid_clock.cc:648] HybridClock initialized: now 1786395827376843 us; error 34 us; skew 500 ppm
Aug 10 21:03:47 dist-test-slave-1x4g krb5kdc[5733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:47.379142 5746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:47.379590 5746 webserver.cc:460] Webserver started at http://127.0.0.1:43587/ using document root <none> and password file <none>
I20260810 21:03:47.379797 5746 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:47.379861 5746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:47.380010 5746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:47.381058 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "c0baade6c23347b3bd1c1586c41baeb7"
format_stamp: "Formatted at 2026-08-10 21:03:47 on dist-test-slave-1x4g"
server_key: "fc427ba05334c3d70eca397109c31a78"
server_key_iv: "546b3fb4d4f680423df7b9b9006bc133"
server_key_version: "encryptionkey@0"
I20260810 21:03:47.381412 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "c0baade6c23347b3bd1c1586c41baeb7"
format_stamp: "Formatted at 2026-08-10 21:03:47 on dist-test-slave-1x4g"
server_key: "fc427ba05334c3d70eca397109c31a78"
server_key_iv: "546b3fb4d4f680423df7b9b9006bc133"
server_key_version: "encryptionkey@0"
I20260810 21:03:47.382771 5746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:47.383471 5765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:47.383635 5746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:47.383738 5746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "c0baade6c23347b3bd1c1586c41baeb7"
format_stamp: "Formatted at 2026-08-10 21:03:47 on dist-test-slave-1x4g"
server_key: "fc427ba05334c3d70eca397109c31a78"
server_key_iv: "546b3fb4d4f680423df7b9b9006bc133"
server_key_version: "encryptionkey@0"
I20260810 21:03:47.383809 5746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:47.386889 5746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:47.388715 5746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:47.388885 5746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:47.392968 5746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37799
I20260810 21:03:47.393023 5822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37799 every 8 connection(s)
I20260810 21:03:47.393406 5746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:47.394236 5823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:47.396227 5823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7: Bootstrap starting.
I20260810 21:03:47.396826 5823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:47.397158 5823 log.cc:824] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:47.397794 5823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7: No bootstrap required, opened a new log
I20260810 21:03:47.399268 5823 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } }
I20260810 21:03:47.399389 5823 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:47.399410 5823 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0baade6c23347b3bd1c1586c41baeb7, State: Initialized, Role: FOLLOWER
I20260810 21:03:47.399593 5823 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } }
I20260810 21:03:47.399675 5823 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:47.399716 5823 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:47.399765 5823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:47.400264 5823 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } }
I20260810 21:03:47.400367 5823 leader_election.cc:302] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0baade6c23347b3bd1c1586c41baeb7; no voters:
I20260810 21:03:47.400557 5823 leader_election.cc:288] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:47.400610 5827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:47.400771 5827 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [term 1 LEADER]: Becoming Leader. State: Replica: c0baade6c23347b3bd1c1586c41baeb7, State: Running, Role: LEADER
I20260810 21:03:47.400826 5823 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:47.400884 5827 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } }
I20260810 21:03:47.401281 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0baade6c23347b3bd1c1586c41baeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } } }
I20260810 21:03:47.401365 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:47.401430 5829 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0baade6c23347b3bd1c1586c41baeb7. Latest consensus state: current_term: 1 leader_uuid: "c0baade6c23347b3bd1c1586c41baeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0baade6c23347b3bd1c1586c41baeb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37799 } } }
I20260810 21:03:47.401538 5829 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:47.401595 5836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:47.402158 5836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:47.403108 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5746
I20260810 21:03:47.403220 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:47.403402 15136 external_mini_cluster.cc:1428] Setting key d668518a791ee9fd24e0135b23e93052
I20260810 21:03:47.403962 5836 catalog_manager.cc:1347] Generated new cluster ID: 942b9c3a305a4fe6897999c15a646573
I20260810 21:03:47.404016 5836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:47.406328 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:47.403803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:54785 (local address 127.0.0.1:37799)
0810 21:03:47.404905 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:47.404913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:47.404941 (+ 28us) server_negotiation.cc:407] Connection header received
0810 21:03:47.405041 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:47.405046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:47.405068 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:47.405201 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:47.405866 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:47.405870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:47.405870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:47.405873 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:47.405948 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:47.405957 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:47.406054 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:47.406120 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 21:03:47.406169 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":130,"threads_started":1}
I20260810 21:03:47.407163 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:47.409832 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:47.408493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:51745 (local address 127.0.0.1:37799)
0810 21:03:47.408527 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:47.408529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:47.408938 (+ 409us) server_negotiation.cc:407] Connection header received
0810 21:03:47.409019 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:47.409020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:47.409032 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:47.409079 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:47.409643 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:47.409645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:47.409645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:47.409649 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:47.409661 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:47.409664 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:47.409709 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:47.409759 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 21:03:47.409774 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:03:47.424448 5836 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:47.424593 5836 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:47.436576 5836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0baade6c23347b3bd1c1586c41baeb7: Generated new TSK 0
I20260810 21:03:47.436757 5836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:47.445272 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5746
2026-08-10T21:03:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:47 dist-test-slave-1x4g krb5kdc[5860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:47 dist-test-slave-1x4g krb5kdc[5860](info): set up 2 sockets
Aug 10 21:03:47 dist-test-slave-1x4g krb5kdc[5860](info): commencing operation
krb5kdc: starting...
W20260810 21:03:48.497591 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:48.613615 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:48.693006 5879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:48.693210 5879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:48.693244 5879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:48.694330 5879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:48.694393 5879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:48.694418 5879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:48.694451 5879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:48.694481 5879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:48.695927 5879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:48.696192 5879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:48.696410 5879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:48.700878 5887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:48.700937 5889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:48.700886 5884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:48.701162 5879 server_base.cc:1034] running on GCE node
I20260810 21:03:48.701454 5879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:48.701709 5879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:48.702853 5879 hybrid_clock.cc:648] HybridClock initialized: now 1786395828702841 us; error 27 us; skew 500 ppm
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:48.709465 5879 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:48.709940 5879 webserver.cc:460] Webserver started at http://127.0.0.1:40251/ using document root <none> and password file <none>
I20260810 21:03:48.710171 5879 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:48.710214 5879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:48.710372 5879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:48.711318 5879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "a2cc5c4350394121a55b2a3e68d39e13"
format_stamp: "Formatted at 2026-08-10 21:03:48 on dist-test-slave-1x4g"
server_key: "9222c747797ec683c04c475daceb75d6"
server_key_iv: "da236aab80c2332d20dc48460659217e"
server_key_version: "encryptionkey@0"
I20260810 21:03:48.711683 5879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "a2cc5c4350394121a55b2a3e68d39e13"
format_stamp: "Formatted at 2026-08-10 21:03:48 on dist-test-slave-1x4g"
server_key: "9222c747797ec683c04c475daceb75d6"
server_key_iv: "da236aab80c2332d20dc48460659217e"
server_key_version: "encryptionkey@0"
I20260810 21:03:48.713019 5879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:03:48.713739 5896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:48.713930 5879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:48.714020 5879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "a2cc5c4350394121a55b2a3e68d39e13"
format_stamp: "Formatted at 2026-08-10 21:03:48 on dist-test-slave-1x4g"
server_key: "9222c747797ec683c04c475daceb75d6"
server_key_iv: "da236aab80c2332d20dc48460659217e"
server_key_version: "encryptionkey@0"
I20260810 21:03:48.714092 5879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:48.735481 5879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:48.737236 5879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:48.737385 5879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:48.741673 5879 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36593
I20260810 21:03:48.741721 5956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36593 every 8 connection(s)
I20260810 21:03:48.742065 5879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:48.742892 5957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:48.745085 5957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13: Bootstrap starting.
I20260810 21:03:48.745643 5957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:48.745937 5957 log.cc:824] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:48.746596 5957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13: No bootstrap required, opened a new log
I20260810 21:03:48.748135 5957 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 21:03:48.748308 5957 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:48.748356 5957 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cc5c4350394121a55b2a3e68d39e13, State: Initialized, Role: FOLLOWER
I20260810 21:03:48.748438 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 5879
I20260810 21:03:48.748574 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:48.748570 5957 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 21:03:48.748679 5957 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:48.748724 5957 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:48.748776 15136 external_mini_cluster.cc:1428] Setting key b808ed6d5354eca9ea666d7786c15ffc
I20260810 21:03:48.748797 5957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:48.749413 5957 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 21:03:48.749531 5957 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2cc5c4350394121a55b2a3e68d39e13; no voters:
I20260810 21:03:48.749681 5957 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:48.749724 5962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:48.749872 5962 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [term 1 LEADER]: Becoming Leader. State: Replica: a2cc5c4350394121a55b2a3e68d39e13, State: Running, Role: LEADER
I20260810 21:03:48.749934 5957 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:48.750002 5962 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } }
I20260810 21:03:48.750560 5963 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2cc5c4350394121a55b2a3e68d39e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } } }
I20260810 21:03:48.750653 5963 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:48.750928 5965 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2cc5c4350394121a55b2a3e68d39e13. Latest consensus state: current_term: 1 leader_uuid: "a2cc5c4350394121a55b2a3e68d39e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2cc5c4350394121a55b2a3e68d39e13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36593 } } }
I20260810 21:03:48.750998 5965 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:48.751362 5975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:48.751991 5975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:48.753643 5975 catalog_manager.cc:1347] Generated new cluster ID: 61700bfcb35a4747a2b518f51bf01b98
I20260810 21:03:48.753698 5975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 21:03:48.758127 5975 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:48.758780 5975 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:48.760085 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:48.749308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:38021 (local address 127.0.0.1:36593)
0810 21:03:48.749501 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:48.749510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:48.749542 (+ 32us) server_negotiation.cc:407] Connection header received
0810 21:03:48.749673 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:48.749680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:48.749741 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:48.750635 (+ 894us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:48.750754 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:48.752275 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:48.753287 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:48.753726 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:48.753894 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:48.756154 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:48.756158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:48.756158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:48.756169 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:48.758562 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:48.758946 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:48.758949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:48.758949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:48.759049 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:48.759284 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:48.759285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:48.759286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:48.759515 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:48.759690 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:48.759813 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 21:03:48.759909 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260810 21:03:48.760947 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:48.768787 5975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2cc5c4350394121a55b2a3e68d39e13: Generated new TSK 0
I20260810 21:03:48.768962 5975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:48.770995 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:48.765103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:53005 (local address 127.0.0.1:36593)
0810 21:03:48.765140 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:48.765141 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:48.765265 (+ 124us) server_negotiation.cc:407] Connection header received
0810 21:03:48.765332 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:48.765333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:48.765343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:48.765393 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:48.765860 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:48.766313 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:48.766965 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:48.767189 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:48.767259 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:48.768532 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:48.768534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:48.768534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:48.768541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:48.769925 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:48.770291 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:48.770292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:48.770292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:48.770345 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:48.770555 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:48.770556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:48.770557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:48.770651 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:48.770748 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:48.770854 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 21:03:48.770912 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:03:48.773006 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 5879
W20260810 21:03:48.782060 5858 connection.cc:535] client connection to 127.0.0.1:36593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36593): Connection reset by peer (error 104)
2026-08-10T21:03:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5994](info): set up 2 sockets
Aug 10 21:03:48 dist-test-slave-1x4g krb5kdc[5994](info): commencing operation
krb5kdc: starting...
W20260810 21:03:49.833798 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:49 dist-test-slave-1x4g krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:49.946828 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:50.026930 6010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:50.027194 6010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:50.027235 6010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:50.028337 6010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:50.028415 6010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:50.028442 6010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:50.028472 6010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:50.028496 6010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:50.029953 6010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:50.030231 6010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:50.030457 6010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:50.035233 6017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:50.035281 6016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:50.035384 6010 server_base.cc:1034] running on GCE node
W20260810 21:03:50.035234 6019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:50.035717 6010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:50.035998 6010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:50.037160 6010 hybrid_clock.cc:648] HybridClock initialized: now 1786395830037147 us; error 43 us; skew 500 ppm
Aug 10 21:03:50 dist-test-slave-1x4g krb5kdc[5994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:50.039630 6010 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:50.040088 6010 webserver.cc:460] Webserver started at http://127.0.0.1:46235/ using document root <none> and password file <none>
I20260810 21:03:50.040318 6010 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:50.040383 6010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:50.040536 6010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:50.041607 6010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "b1fc1e56c0964b83b558ed2445d4fa67"
format_stamp: "Formatted at 2026-08-10 21:03:50 on dist-test-slave-1x4g"
server_key: "5c59a99d3b8f476d683ac088c90c273f"
server_key_iv: "32591bb24627ea244f4535ebe3bafa01"
server_key_version: "encryptionkey@0"
I20260810 21:03:50.041968 6010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "b1fc1e56c0964b83b558ed2445d4fa67"
format_stamp: "Formatted at 2026-08-10 21:03:50 on dist-test-slave-1x4g"
server_key: "5c59a99d3b8f476d683ac088c90c273f"
server_key_iv: "32591bb24627ea244f4535ebe3bafa01"
server_key_version: "encryptionkey@0"
I20260810 21:03:50.043269 6010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 21:03:50.043977 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:50.044140 6010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:03:50.044238 6010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "b1fc1e56c0964b83b558ed2445d4fa67"
format_stamp: "Formatted at 2026-08-10 21:03:50 on dist-test-slave-1x4g"
server_key: "5c59a99d3b8f476d683ac088c90c273f"
server_key_iv: "32591bb24627ea244f4535ebe3bafa01"
server_key_version: "encryptionkey@0"
I20260810 21:03:50.044322 6010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:50.050493 6010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:50.052738 6010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:50.052906 6010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:50.056994 6010 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46693
I20260810 21:03:50.057024 6083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46693 every 8 connection(s)
I20260810 21:03:50.057353 6010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:50.058205 6084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:50.060302 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67: Bootstrap starting.
I20260810 21:03:50.060918 6084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:50.061273 6084 log.cc:824] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:50.061314 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6010
I20260810 21:03:50.061416 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:50.061646 15136 external_mini_cluster.cc:1428] Setting key 767383b711a56d474210eaa2e3260d15
I20260810 21:03:50.062085 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67: No bootstrap required, opened a new log
I20260810 21:03:50.063741 6084 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 21:03:50.063895 6084 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:50.063959 6084 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1fc1e56c0964b83b558ed2445d4fa67, State: Initialized, Role: FOLLOWER
I20260810 21:03:50.064098 6084 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 21:03:50.064196 6084 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:50.064240 6084 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:50.064287 6084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:50.064845 6084 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 21:03:50.064971 6084 leader_election.cc:302] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1fc1e56c0964b83b558ed2445d4fa67; no voters:
I20260810 21:03:50.065171 6084 leader_election.cc:288] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:50.065201 6089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:50.065402 6089 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [term 1 LEADER]: Becoming Leader. State: Replica: b1fc1e56c0964b83b558ed2445d4fa67, State: Running, Role: LEADER
I20260810 21:03:50.065431 6084 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:50.065539 6089 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } }
I20260810 21:03:50.065917 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1fc1e56c0964b83b558ed2445d4fa67. Latest consensus state: current_term: 1 leader_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260810 21:03:50.066032 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:50.066270 6098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:50.066787 6098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:50.067708 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1fc1e56c0964b83b558ed2445d4fa67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46693 } } }
I20260810 21:03:50.067790 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:50.068504 6098 catalog_manager.cc:1347] Generated new cluster ID: 0095dfdea2ee4d92a7c45724de02d2d4
I20260810 21:03:50.068558 6098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:50.069284 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:50.062113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:53503 (local address 127.0.0.1:46693)
0810 21:03:50.062291 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:50.062300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:50.062333 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:03:50.063729 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:50.063735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:50.063780 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:50.063910 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:50.064151 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:50.065687 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:50.066901 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:50.067476 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:50.067566 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:50.068570 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:50.068573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:50.068574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:50.068771 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:50.068828 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:50.068841 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:50.068927 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:50.069049 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 21:03:50.069073 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260810 21:03:50.070201 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:50.075655 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:50.071755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:50145 (local address 127.0.0.1:46693)
0810 21:03:50.071829 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:50.071831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:50.072041 (+ 210us) server_negotiation.cc:407] Connection header received
0810 21:03:50.072158 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:50.072158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:50.072172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:50.072234 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:50.072496 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:50.073408 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:50.074379 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:50.074719 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:50.074793 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:50.075385 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:50.075386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:50.075386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:50.075390 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:50.075403 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:50.075408 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:50.075495 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:50.075571 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 21:03:50.075584 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:50.081859 6098 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:50.082314 6098 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:50.086520 6098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1fc1e56c0964b83b558ed2445d4fa67: Generated new TSK 0
I20260810 21:03:50.086671 6098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:50.091889 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6010
W20260810 21:03:50.099499 5992 connection.cc:535] client connection to 127.0.0.1:46693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46693): Connection reset by peer (error 104)
2026-08-10T21:03:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:50 dist-test-slave-1x4g krb5kdc[6116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:50 dist-test-slave-1x4g krb5kdc[6116](info): set up 2 sockets
Aug 10 21:03:50 dist-test-slave-1x4g krb5kdc[6116](info): commencing operation
krb5kdc: starting...
W20260810 21:03:51.154553 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:51 dist-test-slave-1x4g krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 21:03:51.267110 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:51.346040 6131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:51.346244 6131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:51.346280 6131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:51.347350 6131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:51.347411 6131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:51.347434 6131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:51.347484 6131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:51.347508 6131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:51.348937 6131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33597
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:51.349184 6131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:51.349395 6131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:51.354316 6139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:51.354317 6136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:51.354393 6131 server_base.cc:1034] running on GCE node
W20260810 21:03:51.354331 6137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:51.354737 6131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:51.355028 6131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:51.356190 6131 hybrid_clock.cc:648] HybridClock initialized: now 1786395831356177 us; error 26 us; skew 500 ppm
Aug 10 21:03:51 dist-test-slave-1x4g krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:51.358603 6131 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 21:03:51.359033 6131 webserver.cc:460] Webserver started at http://127.0.0.1:39525/ using document root <none> and password file <none>
I20260810 21:03:51.359246 6131 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:51.359314 6131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:51.359457 6131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:51.360399 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "a525adbe92144193aa2c138cd541bc69"
format_stamp: "Formatted at 2026-08-10 21:03:51 on dist-test-slave-1x4g"
server_key: "12b7d299366ff0334d2d2d4b81873f5b"
server_key_iv: "21a6debbff2d19593ea9691f6ac78881"
server_key_version: "encryptionkey@0"
I20260810 21:03:51.360810 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "a525adbe92144193aa2c138cd541bc69"
format_stamp: "Formatted at 2026-08-10 21:03:51 on dist-test-slave-1x4g"
server_key: "12b7d299366ff0334d2d2d4b81873f5b"
server_key_iv: "21a6debbff2d19593ea9691f6ac78881"
server_key_version: "encryptionkey@0"
I20260810 21:03:51.362126 6131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:03:51.362840 6146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:51.363018 6131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:51.363117 6131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "a525adbe92144193aa2c138cd541bc69"
format_stamp: "Formatted at 2026-08-10 21:03:51 on dist-test-slave-1x4g"
server_key: "12b7d299366ff0334d2d2d4b81873f5b"
server_key_iv: "21a6debbff2d19593ea9691f6ac78881"
server_key_version: "encryptionkey@0"
I20260810 21:03:51.363196 6131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:51.366422 6131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:51.368647 6131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:51.368861 6131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:51.372891 6131 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33597
I20260810 21:03:51.372937 6209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33597 every 8 connection(s)
I20260810 21:03:51.373257 6131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:51.374099 6210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:51.376247 6210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: Bootstrap starting.
I20260810 21:03:51.376926 6210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:51.377220 6210 log.cc:824] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:51.377885 6210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: No bootstrap required, opened a new log
I20260810 21:03:51.379489 6210 raft_consensus.cc:348] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } }
I20260810 21:03:51.379616 6210 raft_consensus.cc:374] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:51.379638 6210 raft_consensus.cc:729] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a525adbe92144193aa2c138cd541bc69, State: Initialized, Role: FOLLOWER
I20260810 21:03:51.379822 6210 consensus_queue.cc:260] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } }
I20260810 21:03:51.379957 6210 raft_consensus.cc:388] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:51.380013 6210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:51.380069 6210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:51.380599 6210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } }
I20260810 21:03:51.380707 6210 leader_election.cc:302] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a525adbe92144193aa2c138cd541bc69; no voters:
I20260810 21:03:51.380909 6210 leader_election.cc:288] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:51.380954 6213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:51.381135 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6131
I20260810 21:03:51.381152 6213 raft_consensus.cc:686] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [term 1 LEADER]: Becoming Leader. State: Replica: a525adbe92144193aa2c138cd541bc69, State: Running, Role: LEADER
I20260810 21:03:51.381183 6210 sys_catalog.cc:564] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:51.381286 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:51.381268 6213 consensus_queue.cc:237] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } }
I20260810 21:03:51.381482 15136 external_mini_cluster.cc:1428] Setting key 389df8b31c45da1967070761abad1571
I20260810 21:03:51.381676 6213 sys_catalog.cc:455] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a525adbe92144193aa2c138cd541bc69. Latest consensus state: current_term: 1 leader_uuid: "a525adbe92144193aa2c138cd541bc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } } }
I20260810 21:03:51.381762 6213 sys_catalog.cc:458] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:51.381860 6214 sys_catalog.cc:455] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a525adbe92144193aa2c138cd541bc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a525adbe92144193aa2c138cd541bc69" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33597 } } }
I20260810 21:03:51.381956 6214 sys_catalog.cc:458] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69 [sys.catalog]: This master's current role is: LEADER
W20260810 21:03:51.383378 6231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:03:51.383446 6231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:03:51.383567 6221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:51.383750 6221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:51.384127 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:51.382190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:54707 (local address 127.0.0.1:33597)
0810 21:03:51.382374 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:51.382384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:51.382487 (+ 103us) server_negotiation.cc:407] Connection header received
0810 21:03:51.382742 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:51.382750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:51.382783 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:51.382918 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:51.383593 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:51.383597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:51.383597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:51.383602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:51.383644 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:51.383659 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:51.383725 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:51.383802 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 21:03:51.383943 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260810 21:03:51.385222 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:51.385318 6221 catalog_manager.cc:1347] Generated new cluster ID: d28f0b13fc554b5986056f80ac0b1d13
I20260810 21:03:51.385360 6221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:51.387873 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:51.386651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:60171 (local address 127.0.0.1:33597)
0810 21:03:51.386709 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:51.386711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:51.386910 (+ 199us) server_negotiation.cc:407] Connection header received
0810 21:03:51.386988 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:51.386989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:51.386994 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:51.387049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:51.387641 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:51.387642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:51.387642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:51.387646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:51.387661 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:51.387665 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:51.387720 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:51.387776 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 21:03:51.387799 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:03:51.393843 6221 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:51.393985 6221 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:51.407131 6221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a525adbe92144193aa2c138cd541bc69: Generated new TSK 0
I20260810 21:03:51.407333 6221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:51.414768 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6131
2026-08-10T21:03:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:51 dist-test-slave-1x4g krb5kdc[6241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:51 dist-test-slave-1x4g krb5kdc[6241](info): set up 2 sockets
Aug 10 21:03:51 dist-test-slave-1x4g krb5kdc[6241](info): commencing operation
krb5kdc: starting...
W20260810 21:03:52.482229 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:52.597641 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45273
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:45273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:52.676018 6255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:52.676203 6255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:52.676225 6255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:52.677426 6255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:52.677479 6255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:52.677492 6255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:52.677541 6255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:52.677567 6255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:52.679000 6255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:45273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:52.679253 6255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:52.679476 6255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:52.684134 6263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:52.684126 6261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:52.684128 6260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:52.684191 6255 server_base.cc:1034] running on GCE node
I20260810 21:03:52.684522 6255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:52.684757 6255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:52.685973 6255 hybrid_clock.cc:648] HybridClock initialized: now 1786395832685953 us; error 39 us; skew 500 ppm
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395832, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:52.688335 6255 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:52.688868 6255 webserver.cc:460] Webserver started at http://127.14.200.62:43627/ using document root <none> and password file <none>
I20260810 21:03:52.689105 6255 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:52.689170 6255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:52.689299 6255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:52.690267 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "3e3bcc0a7250438093aaa3cc70810de6"
format_stamp: "Formatted at 2026-08-10 21:03:52 on dist-test-slave-1x4g"
server_key: "f03c0bbd6ac1259df106bbb528a913d7"
server_key_iv: "9d1e0c0a0c509fecd2bd750f9ef98396"
server_key_version: "encryptionkey@0"
I20260810 21:03:52.690619 6255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "3e3bcc0a7250438093aaa3cc70810de6"
format_stamp: "Formatted at 2026-08-10 21:03:52 on dist-test-slave-1x4g"
server_key: "f03c0bbd6ac1259df106bbb528a913d7"
server_key_iv: "9d1e0c0a0c509fecd2bd750f9ef98396"
server_key_version: "encryptionkey@0"
I20260810 21:03:52.691928 6255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:52.692647 6271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:52.692860 6255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:52.692970 6255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "3e3bcc0a7250438093aaa3cc70810de6"
format_stamp: "Formatted at 2026-08-10 21:03:52 on dist-test-slave-1x4g"
server_key: "f03c0bbd6ac1259df106bbb528a913d7"
server_key_iv: "9d1e0c0a0c509fecd2bd750f9ef98396"
server_key_version: "encryptionkey@0"
I20260810 21:03:52.693051 6255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:52.712692 6255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:52.714488 6255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:52.714644 6255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:52.718604 6255 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:45273
I20260810 21:03:52.718637 6327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:45273 every 8 connection(s)
I20260810 21:03:52.718963 6255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:52.719785 6328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:52.721889 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6: Bootstrap starting.
I20260810 21:03:52.722239 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6255
I20260810 21:03:52.722370 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:52.722595 15136 external_mini_cluster.cc:1428] Setting key da16219740eb0fb7db2c919f028339fd
I20260810 21:03:52.722606 6328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:52.722931 6328 log.cc:824] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:52.723670 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6: No bootstrap required, opened a new log
I20260810 21:03:52.725894 6328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } }
I20260810 21:03:52.726058 6328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:52.726089 6328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3bcc0a7250438093aaa3cc70810de6, State: Initialized, Role: FOLLOWER
I20260810 21:03:52.726213 6328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } }
I20260810 21:03:52.726274 6328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:52.726300 6328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:52.726341 6328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:52.726926 6328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } }
I20260810 21:03:52.727044 6328 leader_election.cc:302] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e3bcc0a7250438093aaa3cc70810de6; no voters:
I20260810 21:03:52.727216 6328 leader_election.cc:288] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:52.727254 6333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:52.727396 6333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3bcc0a7250438093aaa3cc70810de6, State: Running, Role: LEADER
I20260810 21:03:52.727475 6333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } }
I20260810 21:03:52.727528 6328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:52.727782 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e3bcc0a7250438093aaa3cc70810de6. Latest consensus state: current_term: 1 leader_uuid: "3e3bcc0a7250438093aaa3cc70810de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } } }
I20260810 21:03:52.727881 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:52.728144 6338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:52.728646 6338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:52.730078 6334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e3bcc0a7250438093aaa3cc70810de6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3bcc0a7250438093aaa3cc70810de6" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 45273 } } }
I20260810 21:03:52.730218 6334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:52.731022 6338 catalog_manager.cc:1347] Generated new cluster ID: b6f2d5c692a440118a7e137bb0f537e5
I20260810 21:03:52.731074 6338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:52.733335 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:52.723114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59503 (local address 127.14.200.62:45273)
0810 21:03:52.723273 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:52.723282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:52.723317 (+ 35us) server_negotiation.cc:407] Connection header received
0810 21:03:52.723809 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:52.723816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:52.723862 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:52.723969 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:52.724735 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:52.725561 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:52.726567 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:52.726962 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:52.727112 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:52.729910 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:52.729913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:52.729914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:52.729926 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:52.731728 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:52.732121 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:52.732124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:52.732125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:52.732220 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:52.732474 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:52.732475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:52.732475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:52.732703 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:52.732925 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:52.733034 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 21:03:52.733136 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260810 21:03:52.734478 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:52.741909 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:52.735773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42163 (local address 127.14.200.62:45273)
0810 21:03:52.735922 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:52.735924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:52.735934 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:03:52.735954 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:52.735955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:52.735975 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:52.736051 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:52.736497 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:52.737090 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:52.737772 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:52.737998 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:52.738065 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:52.739207 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:52.739208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:52.739209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:52.739215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:52.740588 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:52.741032 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:52.741034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:52.741034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:52.741090 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:52.741323 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:52.741324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:52.741325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:52.741438 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:52.741559 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:52.741681 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 21:03:52.741741 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260810 21:03:52.744971 6338 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:52.745605 6338 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:52.750146 6338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e3bcc0a7250438093aaa3cc70810de6: Generated new TSK 0
I20260810 21:03:52.750305 6338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:52.754375 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6255
W20260810 21:03:52.765492 6239 connection.cc:535] client connection to 127.14.200.62:45273 recv error: Network error: failed to read from TLS socket (remote: 127.14.200.62:45273): Connection reset by peer (error 104)
2026-08-10T21:03:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6361](info): set up 2 sockets
Aug 10 21:03:52 dist-test-slave-1x4g krb5kdc[6361](info): commencing operation
krb5kdc: starting...
W20260810 21:03:53.808709 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:53 dist-test-slave-1x4g krb5kdc[6361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:53.922775 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:44115
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:32807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:44115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:54.000193 6377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:54.000415 6377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:54.000451 6377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:54.001539 6377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:54.001596 6377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:54.001619 6377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:54.001639 6377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:54.001663 6377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:54.003142 6377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:32807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:44115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:44115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:54.003381 6377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:54.003618 6377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:54.008340 6385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:54.008379 6383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:54.008572 6382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:54.008706 6377 server_base.cc:1034] running on GCE node
I20260810 21:03:54.008970 6377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:54.009320 6377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:54.010434 6377 hybrid_clock.cc:648] HybridClock initialized: now 1786395834010415 us; error 31 us; skew 500 ppm
Aug 10 21:03:54 dist-test-slave-1x4g krb5kdc[6361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395834, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:54.016932 6377 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:54.017371 6377 webserver.cc:460] Webserver started at http://127.14.200.62:33429/ using document root <none> and password file <none>
I20260810 21:03:54.017627 6377 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:54.017691 6377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:54.017831 6377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:54.018870 6377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "3ffcb0925d154071b5bce6acc18ada7b"
format_stamp: "Formatted at 2026-08-10 21:03:54 on dist-test-slave-1x4g"
server_key: "d4e4c4a179f67a79dc5e37d45675121a"
server_key_iv: "9eb49bd8752ce13212df71142ad00b6c"
server_key_version: "encryptionkey@0"
I20260810 21:03:54.019238 6377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "3ffcb0925d154071b5bce6acc18ada7b"
format_stamp: "Formatted at 2026-08-10 21:03:54 on dist-test-slave-1x4g"
server_key: "d4e4c4a179f67a79dc5e37d45675121a"
server_key_iv: "9eb49bd8752ce13212df71142ad00b6c"
server_key_version: "encryptionkey@0"
I20260810 21:03:54.020535 6377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:54.021288 6392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:54.021447 6377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:54.021519 6377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "3ffcb0925d154071b5bce6acc18ada7b"
format_stamp: "Formatted at 2026-08-10 21:03:54 on dist-test-slave-1x4g"
server_key: "d4e4c4a179f67a79dc5e37d45675121a"
server_key_iv: "9eb49bd8752ce13212df71142ad00b6c"
server_key_version: "encryptionkey@0"
I20260810 21:03:54.021595 6377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:54.033862 6377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:54.036067 6377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:54.036211 6377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:54.040321 6377 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:44115
I20260810 21:03:54.040372 6450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:44115 every 8 connection(s)
I20260810 21:03:54.040704 6377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:54.041544 6452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:54.043768 6452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b: Bootstrap starting.
I20260810 21:03:54.044294 6452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:54.044581 6452 log.cc:824] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:54.045255 6452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b: No bootstrap required, opened a new log
I20260810 21:03:54.046734 6452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } }
I20260810 21:03:54.046877 6452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:54.046947 6452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ffcb0925d154071b5bce6acc18ada7b, State: Initialized, Role: FOLLOWER
I20260810 21:03:54.047102 6452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } }
I20260810 21:03:54.047192 6452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:54.047180 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6377
I20260810 21:03:54.047236 6452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:54.047284 6452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:54.047298 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:54.047477 15136 external_mini_cluster.cc:1428] Setting key feceee8b53dc5053f6741dfe7c5f3830
I20260810 21:03:54.048319 6452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } }
I20260810 21:03:54.048429 6452 leader_election.cc:302] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ffcb0925d154071b5bce6acc18ada7b; no voters:
I20260810 21:03:54.048591 6452 leader_election.cc:288] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:54.048657 6457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:54.048822 6457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [term 1 LEADER]: Becoming Leader. State: Replica: 3ffcb0925d154071b5bce6acc18ada7b, State: Running, Role: LEADER
I20260810 21:03:54.048864 6452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:54.048940 6457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } }
I20260810 21:03:54.049288 6457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ffcb0925d154071b5bce6acc18ada7b. Latest consensus state: current_term: 1 leader_uuid: "3ffcb0925d154071b5bce6acc18ada7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } } }
I20260810 21:03:54.049367 6457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:54.049299 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ffcb0925d154071b5bce6acc18ada7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffcb0925d154071b5bce6acc18ada7b" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 44115 } } }
I20260810 21:03:54.049547 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:54.049566 6464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:54.050143 6464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:54.051916 6464 catalog_manager.cc:1347] Generated new cluster ID: 943e381c44a34961bdc0a81fd9368e95
I20260810 21:03:54.051976 6464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:54.053264 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:54.047928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46001 (local address 127.14.200.62:44115)
0810 21:03:54.048131 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:54.048140 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:54.048173 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:03:54.048278 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:54.048285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:54.048337 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:54.048490 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:54.048776 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:54.050209 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:54.051118 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:54.051703 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:54.051877 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:54.052562 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:54.052566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:54.052567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:54.052769 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:54.052832 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:54.052847 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:54.052947 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:54.053023 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 21:03:54.053055 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260810 21:03:54.054175 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:54.060001 6464 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:54.060134 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:54.055530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39215 (local address 127.14.200.62:44115)
0810 21:03:54.055564 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:54.055566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:54.056958 (+ 1392us) server_negotiation.cc:407] Connection header received
0810 21:03:54.056996 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:54.056997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:54.057011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:54.057080 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:54.057200 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:54.057913 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:54.058886 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:54.059176 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:54.059244 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:54.059849 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:54.059850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:54.059851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:54.059855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:54.059869 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:54.059873 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:54.059928 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:54.060065 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 21:03:54.060077 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:03:54.060662 6464 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:54.067095 6464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ffcb0925d154071b5bce6acc18ada7b: Generated new TSK 0
I20260810 21:03:54.067267 6464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:54.069036 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6377
W20260810 21:03:54.076483 6359 connection.cc:535] client connection to 127.14.200.62:44115 recv error: Network error: failed to read from TLS socket (remote: 127.14.200.62:44115): Connection reset by peer (error 104)
2026-08-10T21:03:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:54 dist-test-slave-1x4g krb5kdc[6488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:54 dist-test-slave-1x4g krb5kdc[6488](info): set up 2 sockets
Aug 10 21:03:54 dist-test-slave-1x4g krb5kdc[6488](info): commencing operation
krb5kdc: starting...
W20260810 21:03:55.127305 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:55 dist-test-slave-1x4g krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:55.241014 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39825
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:55.321055 6504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:55.321254 6504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:55.321275 6504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:55.322439 6504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:55.322489 6504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:55.322506 6504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:55.322558 6504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:55.322592 6504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:55.324076 6504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:39825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:39825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:55.324322 6504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:55.324554 6504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:55.329241 6512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:55.329337 6504 server_base.cc:1034] running on GCE node
W20260810 21:03:55.329247 6509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:55.329252 6510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:55.329705 6504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:55.330035 6504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:55.331211 6504 hybrid_clock.cc:648] HybridClock initialized: now 1786395835331195 us; error 35 us; skew 500 ppm
Aug 10 21:03:55 dist-test-slave-1x4g krb5kdc[6488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395835, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:55.333699 6504 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:55.334050 6504 webserver.cc:460] Webserver started at http://127.14.200.62:35809/ using document root <none> and password file <none>
I20260810 21:03:55.334208 6504 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:55.334244 6504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:55.334322 6504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:55.335307 6504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "deaaf4b10b054d6caabd481640ae6919"
format_stamp: "Formatted at 2026-08-10 21:03:55 on dist-test-slave-1x4g"
server_key: "4091e38ddb65fee043024cb413995246"
server_key_iv: "dfce95f3a4983d31a4b48986cfea0979"
server_key_version: "encryptionkey@0"
I20260810 21:03:55.335608 6504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "deaaf4b10b054d6caabd481640ae6919"
format_stamp: "Formatted at 2026-08-10 21:03:55 on dist-test-slave-1x4g"
server_key: "4091e38ddb65fee043024cb413995246"
server_key_iv: "dfce95f3a4983d31a4b48986cfea0979"
server_key_version: "encryptionkey@0"
I20260810 21:03:55.336899 6504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:55.337594 6519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:55.337759 6504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:55.337831 6504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "deaaf4b10b054d6caabd481640ae6919"
format_stamp: "Formatted at 2026-08-10 21:03:55 on dist-test-slave-1x4g"
server_key: "4091e38ddb65fee043024cb413995246"
server_key_iv: "dfce95f3a4983d31a4b48986cfea0979"
server_key_version: "encryptionkey@0"
I20260810 21:03:55.337882 6504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:55.340998 6504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:55.343061 6504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:55.343160 6504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:55.347088 6504 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:39825
I20260810 21:03:55.347146 6580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:39825 every 8 connection(s)
I20260810 21:03:55.347427 6504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:55.348310 6581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:55.350493 6581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919: Bootstrap starting.
I20260810 21:03:55.351078 6581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:55.351365 6581 log.cc:824] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:55.352054 6581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919: No bootstrap required, opened a new log
I20260810 21:03:55.353752 6581 raft_consensus.cc:348] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } }
I20260810 21:03:55.353904 6581 raft_consensus.cc:374] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:55.353950 6581 raft_consensus.cc:729] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deaaf4b10b054d6caabd481640ae6919, State: Initialized, Role: FOLLOWER
I20260810 21:03:55.354120 6581 consensus_queue.cc:260] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } }
I20260810 21:03:55.354215 6581 raft_consensus.cc:388] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:55.354260 6581 raft_consensus.cc:482] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:55.354310 6581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:55.355109 6581 raft_consensus.cc:504] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } }
I20260810 21:03:55.355222 6581 leader_election.cc:302] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deaaf4b10b054d6caabd481640ae6919; no voters:
I20260810 21:03:55.355228 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6504
I20260810 21:03:55.355377 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:55.355410 6581 leader_election.cc:288] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:55.355446 6586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:55.355620 6586 raft_consensus.cc:686] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [term 1 LEADER]: Becoming Leader. State: Replica: deaaf4b10b054d6caabd481640ae6919, State: Running, Role: LEADER
I20260810 21:03:55.355628 15136 external_mini_cluster.cc:1428] Setting key 6abbc9a7f14fd4ca6928669e39b3786c
I20260810 21:03:55.355664 6581 sys_catalog.cc:564] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:55.355849 6586 consensus_queue.cc:237] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } }
I20260810 21:03:55.357007 6588 sys_catalog.cc:455] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deaaf4b10b054d6caabd481640ae6919. Latest consensus state: current_term: 1 leader_uuid: "deaaf4b10b054d6caabd481640ae6919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } } }
I20260810 21:03:55.357144 6588 sys_catalog.cc:458] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:55.357435 6587 sys_catalog.cc:455] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deaaf4b10b054d6caabd481640ae6919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deaaf4b10b054d6caabd481640ae6919" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 39825 } } }
I20260810 21:03:55.357525 6587 sys_catalog.cc:458] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:55.357547 6604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:55.358250 6604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:55.359022 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:55.356263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58037 (local address 127.14.200.62:39825)
0810 21:03:55.356401 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:55.356411 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:55.356444 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:03:55.356571 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:55.356578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:55.356620 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:55.356768 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:55.357583 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:55.357588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:55.357588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:55.357592 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:55.357631 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:55.357648 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:55.357725 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:55.358787 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0810 21:03:55.358826 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":68,"threads_started":1}
I20260810 21:03:55.359851 6604 catalog_manager.cc:1347] Generated new cluster ID: eb373e015c0c46bdb553c9ecfe0d5873
I20260810 21:03:55.359900 6604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:55.359923 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:55.369851 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:55.365098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45077 (local address 127.14.200.62:39825)
0810 21:03:55.365129 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:55.365130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:55.368956 (+ 3826us) server_negotiation.cc:407] Connection header received
0810 21:03:55.369019 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:55.369020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:55.369025 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:55.369083 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:55.369647 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:55.369648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:55.369648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:55.369652 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:55.369667 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:55.369671 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:55.369724 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:55.369772 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 21:03:55.369783 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 21:03:55.377676 6604 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:55.377835 6604 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:55.381507 6604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deaaf4b10b054d6caabd481640ae6919: Generated new TSK 0
I20260810 21:03:55.381664 6604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:55.384722 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6504
2026-08-10T21:03:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:55 dist-test-slave-1x4g krb5kdc[6616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:55 dist-test-slave-1x4g krb5kdc[6616](info): set up 2 sockets
Aug 10 21:03:55 dist-test-slave-1x4g krb5kdc[6616](info): commencing operation
krb5kdc: starting...
W20260810 21:03:56.444350 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:56.563556 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35113
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:56.646844 6630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:56.647042 6630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:56.647065 6630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:56.648280 6630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:56.648332 6630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:56.648347 6630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:56.648391 6630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:56.648414 6630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:56.649974 6630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:56.650220 6630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:56.650440 6630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:56.655223 6642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:56.655388 6630 server_base.cc:1034] running on GCE node
W20260810 21:03:56.655300 6636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:56.655224 6640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:56.655786 6630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:56.656075 6630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:56.657231 6630 hybrid_clock.cc:648] HybridClock initialized: now 1786395836657222 us; error 34 us; skew 500 ppm
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395836, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:56.659880 6630 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:56.660293 6630 webserver.cc:460] Webserver started at http://127.14.200.62:36027/ using document root <none> and password file <none>
I20260810 21:03:56.660534 6630 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:56.660599 6630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:56.660744 6630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:56.661784 6630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "82b62ebac09240b19f3e6336f2cb0040"
format_stamp: "Formatted at 2026-08-10 21:03:56 on dist-test-slave-1x4g"
server_key: "2fe562d2032550d15ca2175382c30c8e"
server_key_iv: "00240fd1852ab64122d2fdf4313b6ab7"
server_key_version: "encryptionkey@0"
I20260810 21:03:56.662173 6630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "82b62ebac09240b19f3e6336f2cb0040"
format_stamp: "Formatted at 2026-08-10 21:03:56 on dist-test-slave-1x4g"
server_key: "2fe562d2032550d15ca2175382c30c8e"
server_key_iv: "00240fd1852ab64122d2fdf4313b6ab7"
server_key_version: "encryptionkey@0"
I20260810 21:03:56.663482 6630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:03:56.664166 6649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:56.664331 6630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:03:56.664443 6630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "82b62ebac09240b19f3e6336f2cb0040"
format_stamp: "Formatted at 2026-08-10 21:03:56 on dist-test-slave-1x4g"
server_key: "2fe562d2032550d15ca2175382c30c8e"
server_key_iv: "00240fd1852ab64122d2fdf4313b6ab7"
server_key_version: "encryptionkey@0"
I20260810 21:03:56.664516 6630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:56.674533 6630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:56.676746 6630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:56.676926 6630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:56.680858 6630 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:35113
I20260810 21:03:56.680904 6712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:35113 every 8 connection(s)
I20260810 21:03:56.681253 6630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:56.682153 6713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:56.684415 6713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: Bootstrap starting.
I20260810 21:03:56.685034 6713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:56.685320 6713 log.cc:824] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:56.685981 6713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: No bootstrap required, opened a new log
I20260810 21:03:56.687559 6713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } }
I20260810 21:03:56.687693 6713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:56.687753 6713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b62ebac09240b19f3e6336f2cb0040, State: Initialized, Role: FOLLOWER
I20260810 21:03:56.687965 6713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } }
I20260810 21:03:56.688050 6713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:56.688097 6713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:56.688148 6713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:56.688405 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6630
I20260810 21:03:56.688536 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:56.688723 15136 external_mini_cluster.cc:1428] Setting key 05cf48f8290f7afb76883d79a8e926a4
I20260810 21:03:56.689229 6713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } }
I20260810 21:03:56.689399 6713 leader_election.cc:302] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b62ebac09240b19f3e6336f2cb0040; no voters:
I20260810 21:03:56.689602 6713 leader_election.cc:288] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:56.689625 6718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:56.689803 6718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [term 1 LEADER]: Becoming Leader. State: Replica: 82b62ebac09240b19f3e6336f2cb0040, State: Running, Role: LEADER
I20260810 21:03:56.689849 6713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:56.689926 6718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } }
I20260810 21:03:56.690339 6719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b62ebac09240b19f3e6336f2cb0040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } } }
I20260810 21:03:56.690423 6719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:56.690634 6720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b62ebac09240b19f3e6336f2cb0040. Latest consensus state: current_term: 1 leader_uuid: "82b62ebac09240b19f3e6336f2cb0040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b62ebac09240b19f3e6336f2cb0040" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35113 } } }
I20260810 21:03:56.690956 6720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040 [sys.catalog]: This master's current role is: LEADER
W20260810 21:03:56.691792 6737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:03:56.691850 6737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:03:56.691987 6726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:56.692174 6726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:56.693688 6726 catalog_manager.cc:1347] Generated new cluster ID: 773b615493734e7799fc8eac78561ae5
I20260810 21:03:56.693743 6726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:03:56.699438 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:56.689265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59027 (local address 127.14.200.62:35113)
0810 21:03:56.689475 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:56.689484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:56.689522 (+ 38us) server_negotiation.cc:407] Connection header received
0810 21:03:56.689662 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:56.689669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:56.689729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:56.690787 (+ 1058us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:56.690918 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:56.692248 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:56.693333 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:56.693866 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:56.693989 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:56.696230 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:56.696233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:56.696234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:56.696245 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:56.697921 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:56.698322 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:56.698325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:56.698325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:56.698424 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:56.698634 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:56.698635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:56.698635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:56.698848 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:56.699018 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:56.699093 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 21:03:56.699258 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260810 21:03:56.700330 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:56.710680 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:56.701662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33651 (local address 127.14.200.62:35113)
0810 21:03:56.701698 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:56.701700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:56.704947 (+ 3247us) server_negotiation.cc:407] Connection header received
0810 21:03:56.705021 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:56.705022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:56.705032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:56.705080 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:56.705567 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:56.706042 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:56.706724 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:56.706916 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:56.706984 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:56.708095 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:56.708096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:56.708096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:03:56.708103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:56.709553 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:56.709894 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:56.709895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:56.709896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:56.709953 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:03:56.710143 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:03:56.710144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:03:56.710144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:03:56.710251 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:56.710343 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:56.710494 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 21:03:56.710576 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:03:56.735318 6726 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:56.735843 6726 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:56.741739 6726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b62ebac09240b19f3e6336f2cb0040: Generated new TSK 0
I20260810 21:03:56.741921 6726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:56.750702 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6630
2026-08-10T21:03:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6748](info): set up 2 sockets
Aug 10 21:03:56 dist-test-slave-1x4g krb5kdc[6748](info): commencing operation
krb5kdc: starting...
W20260810 21:03:57.814374 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:57 dist-test-slave-1x4g krb5kdc[6748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:57.929555 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36573
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:58.009872 6764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:58.010059 6764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:58.010082 6764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:58.011107 6764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:58.011139 6764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:58.011153 6764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:58.011166 6764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:58.011178 6764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:58.012547 6764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:36573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:58.012724 6764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:58.012951 6764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:58.017629 6771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:58.017637 6764 server_base.cc:1034] running on GCE node
W20260810 21:03:58.017608 6773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:58.017601 6770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:58.018103 6764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:58.018373 6764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:58.019544 6764 hybrid_clock.cc:648] HybridClock initialized: now 1786395838019529 us; error 29 us; skew 500 ppm
Aug 10 21:03:58 dist-test-slave-1x4g krb5kdc[6748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:58.022169 6764 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:58.022569 6764 webserver.cc:460] Webserver started at http://127.14.200.62:37133/ using document root <none> and password file <none>
I20260810 21:03:58.022801 6764 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:58.022864 6764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:58.023005 6764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:58.023988 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "eac27b11e12d49fbb4df2550389c27ef"
format_stamp: "Formatted at 2026-08-10 21:03:58 on dist-test-slave-1x4g"
server_key: "309227059674281e1d66371921c21e84"
server_key_iv: "e3aef6d44df400dd7d428e790aabe1c5"
server_key_version: "encryptionkey@0"
I20260810 21:03:58.024325 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "eac27b11e12d49fbb4df2550389c27ef"
format_stamp: "Formatted at 2026-08-10 21:03:58 on dist-test-slave-1x4g"
server_key: "309227059674281e1d66371921c21e84"
server_key_iv: "e3aef6d44df400dd7d428e790aabe1c5"
server_key_version: "encryptionkey@0"
I20260810 21:03:58.025701 6764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:03:58.026362 6782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:58.026520 6764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:58.026628 6764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "eac27b11e12d49fbb4df2550389c27ef"
format_stamp: "Formatted at 2026-08-10 21:03:58 on dist-test-slave-1x4g"
server_key: "309227059674281e1d66371921c21e84"
server_key_iv: "e3aef6d44df400dd7d428e790aabe1c5"
server_key_version: "encryptionkey@0"
I20260810 21:03:58.026696 6764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:58.041685 6764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:58.043372 6764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:58.043531 6764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:58.047670 6764 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:36573
I20260810 21:03:58.047704 6843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:36573 every 8 connection(s)
I20260810 21:03:58.048098 6764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:58.048902 6844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:58.051064 6844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef: Bootstrap starting.
I20260810 21:03:58.051608 6844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:58.051898 6844 log.cc:824] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:58.052583 6844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef: No bootstrap required, opened a new log
I20260810 21:03:58.054128 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6764
I20260810 21:03:58.054239 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:58.054193 6844 raft_consensus.cc:348] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } }
I20260810 21:03:58.054378 6844 raft_consensus.cc:374] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:58.054430 15136 external_mini_cluster.cc:1428] Setting key 1ab80d2fbc5e0234374c1d330be834ae
I20260810 21:03:58.054425 6844 raft_consensus.cc:729] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac27b11e12d49fbb4df2550389c27ef, State: Initialized, Role: FOLLOWER
I20260810 21:03:58.054550 6844 consensus_queue.cc:260] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } }
I20260810 21:03:58.054648 6844 raft_consensus.cc:388] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:58.054684 6844 raft_consensus.cc:482] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:58.054735 6844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:58.055760 6844 raft_consensus.cc:504] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } }
I20260810 21:03:58.055876 6844 leader_election.cc:302] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eac27b11e12d49fbb4df2550389c27ef; no voters:
I20260810 21:03:58.056038 6844 leader_election.cc:288] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:58.056084 6852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:58.056243 6852 raft_consensus.cc:686] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [term 1 LEADER]: Becoming Leader. State: Replica: eac27b11e12d49fbb4df2550389c27ef, State: Running, Role: LEADER
I20260810 21:03:58.056299 6844 sys_catalog.cc:564] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:58.056372 6852 consensus_queue.cc:237] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } }
I20260810 21:03:58.056828 6852 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader eac27b11e12d49fbb4df2550389c27ef. Latest consensus state: current_term: 1 leader_uuid: "eac27b11e12d49fbb4df2550389c27ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } } }
I20260810 21:03:58.056981 6852 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:58.056836 6853 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eac27b11e12d49fbb4df2550389c27ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac27b11e12d49fbb4df2550389c27ef" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 36573 } } }
I20260810 21:03:58.057163 6853 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:58.057238 6860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:58.057806 6860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:58.059527 6860 catalog_manager.cc:1347] Generated new cluster ID: b0cdea59561848299fcba4ad4ecb4f83
I20260810 21:03:58.059582 6860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:58.060125 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:58.054998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37077 (local address 127.14.200.62:36573)
0810 21:03:58.055141 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:58.055148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:58.055170 (+ 22us) server_negotiation.cc:407] Connection header received
0810 21:03:58.055258 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:58.055263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:58.055299 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:58.055389 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:58.055646 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:58.056853 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:58.058065 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:58.058573 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:58.058681 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:58.059334 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:58.059338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:58.059338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:58.059560 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:58.059599 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:58.059615 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:58.059675 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:58.059864 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 21:03:58.059921 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260810 21:03:58.061091 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:58.065864 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:58.062563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43031 (local address 127.14.200.62:36573)
0810 21:03:58.062592 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:58.062593 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:58.062705 (+ 112us) server_negotiation.cc:407] Connection header received
0810 21:03:58.062765 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:58.062766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:58.062775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:58.062830 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:58.062949 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:58.063657 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:58.064627 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:03:58.064923 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:03:58.064977 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:03:58.065648 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:58.065649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:58.065650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:58.065655 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:58.065666 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:58.065669 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:58.065692 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:58.065775 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 21:03:58.065796 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":14}
I20260810 21:03:58.075249 6860 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:58.075819 6860 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:58.079411 6860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eac27b11e12d49fbb4df2550389c27ef: Generated new TSK 0
I20260810 21:03:58.079573 6860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:58.087011 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6764
2026-08-10T21:03:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:58 dist-test-slave-1x4g krb5kdc[6883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:58 dist-test-slave-1x4g krb5kdc[6883](info): set up 2 sockets
Aug 10 21:03:58 dist-test-slave-1x4g krb5kdc[6883](info): commencing operation
krb5kdc: starting...
W20260810 21:03:59.142179 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:03:59 dist-test-slave-1x4g krb5kdc[6883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:03:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:03:59.256357 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35443
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:42953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:35443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:03:59.335026 6897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:03:59.335218 6897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:03:59.335240 6897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:03:59.336383 6897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:03:59.336423 6897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:03:59.336440 6897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:03:59.336485 6897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:03:59.336512 6897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:03:59.338240 6897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:42953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:35443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:35443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:03:59.338505 6897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:03:59.338747 6897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:03:59.343765 6903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:59.343768 6906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:03:59.343775 6904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:03:59.344027 6897 server_base.cc:1034] running on GCE node
I20260810 21:03:59.344383 6897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:03:59.344810 6897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:03:59.346024 6897 hybrid_clock.cc:648] HybridClock initialized: now 1786395839346001 us; error 36 us; skew 500 ppm
Aug 10 21:03:59 dist-test-slave-1x4g krb5kdc[6883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395839, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:03:59.353048 6897 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:03:59.353461 6897 webserver.cc:460] Webserver started at http://127.14.200.62:44169/ using document root <none> and password file <none>
I20260810 21:03:59.353648 6897 fs_manager.cc:359] Metadata directory not provided
I20260810 21:03:59.353693 6897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:03:59.353790 6897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:03:59.354784 6897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9"
format_stamp: "Formatted at 2026-08-10 21:03:59 on dist-test-slave-1x4g"
server_key: "9dd403be0831d60a6dc3fd6af1c821a5"
server_key_iv: "26acd16c20bf7a58c5d7e2b745da5d51"
server_key_version: "encryptionkey@0"
I20260810 21:03:59.355113 6897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9"
format_stamp: "Formatted at 2026-08-10 21:03:59 on dist-test-slave-1x4g"
server_key: "9dd403be0831d60a6dc3fd6af1c821a5"
server_key_iv: "26acd16c20bf7a58c5d7e2b745da5d51"
server_key_version: "encryptionkey@0"
I20260810 21:03:59.356412 6897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 21:03:59.357218 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:59.357381 6897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:03:59.357489 6897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9"
format_stamp: "Formatted at 2026-08-10 21:03:59 on dist-test-slave-1x4g"
server_key: "9dd403be0831d60a6dc3fd6af1c821a5"
server_key_iv: "26acd16c20bf7a58c5d7e2b745da5d51"
server_key_version: "encryptionkey@0"
I20260810 21:03:59.357565 6897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:03:59.360884 6897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:03:59.363253 6897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:03:59.363422 6897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:03:59.367540 6897 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:35443
I20260810 21:03:59.367569 6970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:35443 every 8 connection(s)
I20260810 21:03:59.367906 6897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:03:59.368916 6971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:03:59.370551 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 6897
I20260810 21:03:59.370683 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:03:59.370884 15136 external_mini_cluster.cc:1428] Setting key b7fe2994221bfc2047e9d740dbe20b8f
I20260810 21:03:59.371238 6971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: Bootstrap starting.
I20260810 21:03:59.372006 6971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: Neither blocks nor log segments found. Creating new log.
I20260810 21:03:59.372352 6971 log.cc:824] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: Log is configured to *not* fsync() on all Append() calls
I20260810 21:03:59.373245 6971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: No bootstrap required, opened a new log
I20260810 21:03:59.373647 6974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:59.371400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54457 (local address 127.14.200.62:35443)
0810 21:03:59.371626 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:59.371636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:59.371673 (+ 37us) server_negotiation.cc:407] Connection header received
0810 21:03:59.372291 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:59.372300 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:59.372335 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:59.372507 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:59.373210 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:59.373214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:59.373214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:59.373219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:59.373263 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:59.373278 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:59.373379 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:59.373417 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 21:03:59.373456 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":116,"threads_started":1}
I20260810 21:03:59.374552 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:03:59.375754 6971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } }
I20260810 21:03:59.375938 6971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 21:03:59.375991 6971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d8f1bf34bc441785ac38aa2b6c1cb9, State: Initialized, Role: FOLLOWER
I20260810 21:03:59.376111 6971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } }
I20260810 21:03:59.376268 6971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 21:03:59.376318 6971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 21:03:59.376372 6971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:03:59.377065 6971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } }
I20260810 21:03:59.377199 6971 leader_election.cc:302] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30d8f1bf34bc441785ac38aa2b6c1cb9; no voters:
I20260810 21:03:59.377367 6971 leader_election.cc:288] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 21:03:59.377389 6982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:03:59.377389 6974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:03:59.376232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60033 (local address 127.14.200.62:35443)
0810 21:03:59.376263 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 21:03:59.376265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:03:59.376383 (+ 118us) server_negotiation.cc:407] Connection header received
0810 21:03:59.376463 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:03:59.376464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:03:59.376469 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:03:59.376532 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:03:59.377153 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:03:59.377154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:03:59.377154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 21:03:59.377158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:03:59.377169 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 21:03:59.377173 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:03:59.377219 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 21:03:59.377302 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 21:03:59.377314 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260810 21:03:59.377621 6971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:03:59.377689 6982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 30d8f1bf34bc441785ac38aa2b6c1cb9, State: Running, Role: LEADER
I20260810 21:03:59.377803 6982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } }
I20260810 21:03:59.378211 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30d8f1bf34bc441785ac38aa2b6c1cb9. Latest consensus state: current_term: 1 leader_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } } }
I20260810 21:03:59.378332 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [sys.catalog]: This master's current role is: LEADER
I20260810 21:03:59.378223 6983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30d8f1bf34bc441785ac38aa2b6c1cb9" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 35443 } } }
I20260810 21:03:59.378463 6983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9 [sys.catalog]: This master's current role is: LEADER
W20260810 21:03:59.379717 7001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:03:59.379774 7001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:03:59.379843 7002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:03:59.380005 7002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:03:59.381214 7002 catalog_manager.cc:1347] Generated new cluster ID: 3b878fcf2a4d439b8adf35c2cb637fdf
I20260810 21:03:59.381265 7002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:03:59.390007 7002 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:03:59.390163 7002 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:03:59.392307 7002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30d8f1bf34bc441785ac38aa2b6c1cb9: Generated new TSK 0
I20260810 21:03:59.392467 7002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 21:03:59.396574 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 6897
2026-08-10T21:03:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1310 ms)
[----------] 18 tests from AuthTokenIssuingTest (23924 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:03:59 dist-test-slave-1x4g krb5kdc[7008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:03:59 dist-test-slave-1x4g krb5kdc[7008](info): set up 2 sockets
Aug 10 21:03:59 dist-test-slave-1x4g krb5kdc[7008](info): commencing operation
krb5kdc: starting...
W20260810 21:04:00.548415 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:04:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:04:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:04:00.672036 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41795
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:04:00.753960 7021 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:00.754170 7021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:00.754189 7021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:00.754240 7021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:00.755344 7021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:00.755385 7021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:00.755399 7021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:00.755411 7021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:00.755460 7021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:00.756949 7021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:41795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:00.757194 7021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:00.757421 7021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:00.762183 7029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:00.762187 7027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:00.762270 7026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:00.762368 7021 server_base.cc:1034] running on GCE node
I20260810 21:04:00.762674 7021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:00.762966 7021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:00.764119 7021 hybrid_clock.cc:648] HybridClock initialized: now 1786395840764109 us; error 37 us; skew 500 ppm
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:00.766614 7021 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:04:00.767117 7021 webserver.cc:460] Webserver started at http://127.14.200.62:41515/ using document root <none> and password file <none>
I20260810 21:04:00.767333 7021 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:00.767405 7021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:00.767570 7021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:00.768610 7021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "573d619267bf4ac68a71354d3192fe0a"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "88318e03b96eccf9f0708a93c3ee637f"
server_key_iv: "acd87dbd7570891a11a7d18a622977bb"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.769017 7021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "573d619267bf4ac68a71354d3192fe0a"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "88318e03b96eccf9f0708a93c3ee637f"
server_key_iv: "acd87dbd7570891a11a7d18a622977bb"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.770319 7021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 21:04:00.770987 7036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:00.771142 7021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:04:00.771245 7021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "573d619267bf4ac68a71354d3192fe0a"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "88318e03b96eccf9f0708a93c3ee637f"
server_key_iv: "acd87dbd7570891a11a7d18a622977bb"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.771310 7021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:00.790515 7021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:00.792862 7021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:00.793030 7021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:00.797181 7021 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:41795
I20260810 21:04:00.797217 7099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:41795 every 8 connection(s)
I20260810 21:04:00.797588 7021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:04:00.798518 7100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:00.800101 7100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:04:00.807569 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7021
I20260810 21:04:00.807695 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:04:00.807893 15136 external_mini_cluster.cc:1428] Setting key a21ba4299344e6d3da5aa0b9e9c44955
I20260810 21:04:00.809453 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.800881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41795 (local address 127.0.0.1:45678)
0810 21:04:00.801132 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.801174 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:00.801669 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:00.801952 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:00.801955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:00.802239 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:00.802389 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.802393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.803414 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.803415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.804068 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.804070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.804552 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.804553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.804614 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.805073 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:00.805081 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:00.806414 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:00.807977 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.807983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.807985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.808172 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.808406 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.808406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.808407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.808488 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.808878 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:00.808884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:00.808947 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 21:04:00.809049 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 21:04:00.809284 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":95,"thread_start_us":134,"threads_started":1}
I20260810 21:04:00.809458 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.800949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.14.200.62:41795)
0810 21:04:00.801134 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:00.801144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:00.801266 (+ 122us) server_negotiation.cc:407] Connection header received
0810 21:04:00.801832 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:00.801834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:00.801852 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:00.801944 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:00.802497 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.803306 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.804159 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.804449 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.804561 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.806500 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:00.806507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:00.806507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:00.806515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:00.807883 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.808261 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.808263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.808264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.808342 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.808552 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.808553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.808553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.808809 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:00.808891 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:00.809085 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 21:04:00.809218 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260810 21:04:00.810137 7100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:00.810480 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.810381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53404)
0810 21:04:00.810419 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.810460 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 21:04:00.810612 7040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
I20260810 21:04:00.811005 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.810916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53414)
0810 21:04:00.810958 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.810983 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 21:04:00.811110 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:33333: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.14.200.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
I20260810 21:04:00.833391 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:33333
--webserver_interface=127.14.200.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:04:00.860334 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } attempt: 1
I20260810 21:04:00.860872 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.860694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53418)
0810 21:04:00.860761 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.860831 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 21:04:00.861243 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.861152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53424)
0810 21:04:00.861192 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.861220 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 21:04:00.861361 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:33333: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
W20260810 21:04:00.914714 7108 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:00.914909 7108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:00.914927 7108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:00.914945 7108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:00.916014 7108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:00.916052 7108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:00.916069 7108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:00.916138 7108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:00.916164 7108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:00.917706 7108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.200.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:00.917946 7108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:00.918152 7108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:00.922801 7114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:00.922812 7117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:00.922811 7115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:00.923045 7108 server_base.cc:1034] running on GCE node
I20260810 21:04:00.923269 7108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:00.923591 7108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:00.924821 7108 hybrid_clock.cc:648] HybridClock initialized: now 1786395840924762 us; error 38 us; skew 500 ppm
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:00.927201 7108 init.cc:377] Logged in from keytab as kudu/127.14.200.61@KRBTEST.COM (short username kudu)
I20260810 21:04:00.927664 7108 webserver.cc:460] Webserver started at http://127.14.200.61:32905/ using document root <none> and password file <none>
I20260810 21:04:00.927876 7108 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:00.927937 7108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:00.928092 7108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:00.929250 7108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data/instance:
uuid: "d62e4188cfae4d5c9c36cbc78c8c6707"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "86bdbbc5834869fd58aa2e8048f1e68c"
server_key_iv: "f73aaad1ba600a1c3356c0a0697c131a"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.929525 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } attempt: 2
I20260810 21:04:00.929627 7108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal/instance:
uuid: "d62e4188cfae4d5c9c36cbc78c8c6707"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "86bdbbc5834869fd58aa2e8048f1e68c"
server_key_iv: "f73aaad1ba600a1c3356c0a0697c131a"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.929927 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.929763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53432)
0810 21:04:00.929834 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.929876 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:04:00.930291 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.930178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53440)
0810 21:04:00.930236 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.930266 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 21:04:00.930404 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:33333: Network error: Client connection negotiation failed: client connection to 127.14.200.61:33333: connect: Connection refused (error 111)
I20260810 21:04:00.931167 7108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 21:04:00.931849 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:00.932022 7108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 21:04:00.932129 7108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "d62e4188cfae4d5c9c36cbc78c8c6707"
format_stamp: "Formatted at 2026-08-10 21:04:00 on dist-test-slave-1x4g"
server_key: "86bdbbc5834869fd58aa2e8048f1e68c"
server_key_iv: "f73aaad1ba600a1c3356c0a0697c131a"
server_key_version: "encryptionkey@0"
I20260810 21:04:00.932204 7108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:00.958909 7108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:00.960881 7108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:00.961031 7108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:00.965204 7108 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.61:33333
I20260810 21:04:00.965260 7187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.61:33333 every 8 connection(s)
I20260810 21:04:00.965597 7108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
I20260810 21:04:00.966547 7188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:00.968045 7188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:00.969722 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7108
I20260810 21:04:00.969836 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-1/wal/instance
I20260810 21:04:00.970036 15136 external_mini_cluster.cc:1428] Setting key ac9791efa96243d7728004aa62dbcca6
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:04:00.977586 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.968716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.14.200.62:41795)
0810 21:04:00.968811 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:00.968814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:00.969011 (+ 197us) server_negotiation.cc:407] Connection header received
0810 21:04:00.969502 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:00.969503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:00.969515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:00.969572 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:00.970308 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.971065 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.972011 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.972248 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.972334 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.974677 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:00.974679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:00.974679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:00.974687 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:00.976095 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.976568 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.976569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.976569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.976628 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.976954 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.976954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.976955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.977060 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:00.977113 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:00.977454 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 21:04:00.977520 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 21:04:00.977867 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.968686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41795 (local address 127.0.0.1:45686)
0810 21:04:00.968888 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.968929 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:00.969402 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:00.969643 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:00.969646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:00.969951 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:00.970202 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.970207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.971155 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.971156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.971920 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.971921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.972373 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.972375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.972491 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.973036 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:00.973045 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:00.974598 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:00.976176 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.976184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.976186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.976482 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.976709 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.976710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.976711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.976872 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.977129 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:00.977137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:00.977310 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 21:04:00.977427 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 21:04:00.977608 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":129,"threads_started":1}
I20260810 21:04:00.978508 7188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.200.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.60@KRBTEST.COM" created.
Aug 10 21:04:00 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:00.986423 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.978806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53452)
0810 21:04:00.978870 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.978883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:00.978939 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:00.979205 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:00.979206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:00.979369 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:00.979418 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.979419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.980283 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.980284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.980825 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:00.980827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.981373 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.981374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:00.981446 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.981870 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:00.981874 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:00.983217 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:00.985265 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.985266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.985266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.985444 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.985674 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:00.985675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:00.985676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:00.985735 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:00.986059 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:00.986060 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:00.986102 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 21:04:00.986181 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 21:04:00.986290 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 21:04:00.986461 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.978873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.14.200.61:33333)
0810 21:04:00.979033 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:00.979037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:00.979060 (+ 23us) server_negotiation.cc:407] Connection header received
0810 21:04:00.979089 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:00.979090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:00.979118 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:00.979191 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:00.979494 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.980167 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.980947 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:00.981270 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:00.981372 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:00.983354 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:00.983365 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:00.983366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:00.983382 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:00.985160 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.985542 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.985545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.985545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.985606 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:00.985822 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:00.985823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:00.985823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:00.985983 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:00.986083 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:00.986213 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 21:04:00.986347 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260810 21:04:00.987217 7188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:00.987605 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.987490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56216)
0810 21:04:00.987538 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.987585 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 21:04:00.987731 7125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
I20260810 21:04:00.988197 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:00.988099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56228)
0810 21:04:00.988146 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:00.988173 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 21:04:00.988306 7188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
I20260810 21:04:00.997386 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:42017
--webserver_interface=127.14.200.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:04:01.021564 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } attempt: 3
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:01.028753 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.022042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53462 (local address 127.14.200.61:33333)
0810 21:04:01.022106 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.022108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.022138 (+ 30us) server_negotiation.cc:407] Connection header received
0810 21:04:01.022175 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.022176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.022188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.022265 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.022611 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.023120 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.023828 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.024084 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.024170 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.026310 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.026311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.026311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.026319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.027671 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.028007 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.028008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.028008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.028067 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.028258 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.028259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.028259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.028367 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.028446 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.028580 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.028656 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:04:01.028753 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.021956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53462)
0810 21:04:01.022021 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.022038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.022101 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.022272 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.022273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.022452 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.022524 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.022526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.023202 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.023203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.023745 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.023747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.024169 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.024170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.024249 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.024709 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.024713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.026214 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.027766 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.027768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.027769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.027940 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.028135 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.028136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.028136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.028195 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.028431 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.028432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.028477 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.028554 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.028655 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 21:04:01.029472 7100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:01.029804 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.029711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56238)
0810 21:04:01.029753 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.029778 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 21:04:01.030113 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.030051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56250)
0810 21:04:01.030079 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.030100 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260810 21:04:01.030200 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
I20260810 21:04:01.037500 7188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 1
I20260810 21:04:01.037905 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.037766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56258)
0810 21:04:01.037833 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.037869 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 21:04:01.038362 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.038154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56270)
0810 21:04:01.038276 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.038309 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260810 21:04:01.038673 7188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
I20260810 21:04:01.061333 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 1
I20260810 21:04:01.061842 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.061700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56286)
0810 21:04:01.061756 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.061793 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 21:04:01.062166 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.062071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56288)
0810 21:04:01.062112 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.062136 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 21:04:01.062273 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
W20260810 21:04:01.096341 7196 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:01.096539 7196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:01.096557 7196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:01.096576 7196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:01.097747 7196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:01.097801 7196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:01.097815 7196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:01.097827 7196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:01.097838 7196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:01.099287 7196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:41795,127.14.200.61:33333,127.14.200.60:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.200.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:01.099495 7196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:01.099664 7196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:01.104437 7206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:01.104496 7202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:01.104452 7204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:01.104629 7196 server_base.cc:1034] running on GCE node
I20260810 21:04:01.104911 7196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:01.105157 7196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:01.106290 7196 hybrid_clock.cc:648] HybridClock initialized: now 1786395841106278 us; error 28 us; skew 500 ppm
I20260810 21:04:01.106845 7188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 2
I20260810 21:04:01.107300 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.107164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56300)
0810 21:04:01.107228 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.107267 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 21:04:01.107645 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.107545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56306)
0810 21:04:01.107590 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.107621 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 21:04:01.107738 7188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395841, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:01.109047 7196 init.cc:377] Logged in from keytab as kudu/127.14.200.60@KRBTEST.COM (short username kudu)
I20260810 21:04:01.109472 7196 webserver.cc:460] Webserver started at http://127.14.200.60:37261/ using document root <none> and password file <none>
I20260810 21:04:01.109700 7196 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:01.109745 7196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:01.109906 7196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:01.110921 7196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data/instance:
uuid: "29df537b84ad42208e95c1717d5c50d0"
format_stamp: "Formatted at 2026-08-10 21:04:01 on dist-test-slave-1x4g"
server_key: "16830594b5c80f31f48dba9a18972ecb"
server_key_iv: "6200e12e92439a56ff1273adf5d11714"
server_key_version: "encryptionkey@0"
I20260810 21:04:01.111259 7196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal/instance:
uuid: "29df537b84ad42208e95c1717d5c50d0"
format_stamp: "Formatted at 2026-08-10 21:04:01 on dist-test-slave-1x4g"
server_key: "16830594b5c80f31f48dba9a18972ecb"
server_key_iv: "6200e12e92439a56ff1273adf5d11714"
server_key_version: "encryptionkey@0"
I20260810 21:04:01.112560 7196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:04:01.113318 7213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:01.113478 7196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:01.113569 7196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "29df537b84ad42208e95c1717d5c50d0"
format_stamp: "Formatted at 2026-08-10 21:04:01 on dist-test-slave-1x4g"
server_key: "16830594b5c80f31f48dba9a18972ecb"
server_key_iv: "6200e12e92439a56ff1273adf5d11714"
server_key_version: "encryptionkey@0"
I20260810 21:04:01.113620 7196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:01.137475 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 2
I20260810 21:04:01.138007 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.137858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56316)
0810 21:04:01.137926 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.137974 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 21:04:01.138383 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.138282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56322)
0810 21:04:01.138330 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.138360 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:04:01.138492 7100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:42017: Network error: Client connection negotiation failed: client connection to 127.14.200.60:42017: connect: Connection refused (error 111)
I20260810 21:04:01.139451 7196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:01.141875 7196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:01.142038 7196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:01.146126 7196 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.60:42017
I20260810 21:04:01.146162 7272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.60:42017 every 8 connection(s)
I20260810 21:04:01.146632 7196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
I20260810 21:04:01.147660 7273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:01.149312 7273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:01.154672 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7196
I20260810 21:04:01.154786 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786395574307397-15136-0/minicluster-data/master-2/wal/instance
I20260810 21:04:01.154991 15136 external_mini_cluster.cc:1428] Setting key 3ca92fbe9fe2251bdea790b032bd04e1
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395841, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:04:01.159320 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.150084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.14.200.62:41795)
0810 21:04:01.150150 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.150153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.150403 (+ 250us) server_negotiation.cc:407] Connection header received
0810 21:04:01.150875 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.150876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.150887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.150929 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.151562 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.152042 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.153056 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.153243 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.153292 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.155661 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.155663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.155664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.155672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.157965 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.158338 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.158339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.158339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.158405 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.158672 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.158673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.158673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.158795 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.158871 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.159150 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.159231 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 21:04:01.159523 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.150079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:41795 (local address 127.0.0.1:45688)
0810 21:04:01.150270 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.150307 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.150781 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.151004 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.151008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.151288 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.151487 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.151491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.152115 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.152116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.152978 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.152981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.153335 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.153336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.153435 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.153905 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.153914 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.155551 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.158042 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.158048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.158049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.158270 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.158468 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.158469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.158469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.158607 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.158867 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.158876 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.159019 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.159126 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.159277 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260810 21:04:01.160116 7273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395841, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:01.164650 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.155399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.14.200.60:42017)
0810 21:04:01.155555 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.155559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.155612 (+ 53us) server_negotiation.cc:407] Connection header received
0810 21:04:01.155691 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.155693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.155705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.155759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.156660 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.157592 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.158547 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.158963 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.159053 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.161792 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.161799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.161800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.161806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.163419 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.163822 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.163825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.163826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.163888 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.164068 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.164069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.164069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.164244 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.164400 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.164447 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.164535 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260810 21:04:01.167975 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.160453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.14.200.61:33333)
0810 21:04:01.160532 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.160534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.160542 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:04:01.160566 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.160567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.160576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.160638 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.160965 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.161456 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.162348 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.162698 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.162767 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.164820 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.164822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.164822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.164829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.166580 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.166963 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.166979 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.166979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.167038 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.167266 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.167268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.167268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.167425 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.167497 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.167752 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.167849 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 21:04:01.168316 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.160362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:33333 (local address 127.0.0.1:53468)
0810 21:04:01.160404 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.160414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.160454 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.160642 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.160643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.160818 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.160882 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.160885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.161536 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.161538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.162237 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.162240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.162774 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.162775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.162844 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.163199 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.163203 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.164679 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.166671 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.166672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.166673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.166902 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.167129 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.167130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.167131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.167193 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.167532 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.167533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.167617 (+ 84us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.168106 (+ 489us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.168204 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 21:04:01.168749 7273 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395841, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:01.175570 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.165692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.14.200.62:41795)
0810 21:04:01.166327 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.166329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.166337 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:04:01.166358 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.166359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.166367 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.166423 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.167119 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.167934 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.169103 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.169391 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.169436 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.172322 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.172323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.172323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.172330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.174052 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.174507 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.174509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.174509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.174571 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.174773 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.174774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.174774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.174904 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.175072 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.175362 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.175449 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611}
I20260810 21:04:01.177731 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.169055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56340)
0810 21:04:01.169103 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.169113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.169146 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.169506 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.169506 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.169983 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.170045 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.170048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.170958 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.170959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.171419 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.171420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.171881 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.171882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.171930 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.173297 (+ 1367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.173302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.174794 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.176751 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.176751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.176752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.176955 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.177152 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.177153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.177154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.177217 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.177407 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.177408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.177449 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.177528 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.177620 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:04:01.177999 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.169295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56340 (local address 127.14.200.60:42017)
0810 21:04:01.169353 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.169355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.169362 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:04:01.169380 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.169381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.169391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.169475 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.170128 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.170874 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.171494 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.171812 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.171873 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.174904 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.174905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.174905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.174912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.176676 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.177039 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.177044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.177044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.177096 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.177275 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.177275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.177275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.177358 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.177398 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.177543 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.177660 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":19}
I20260810 21:04:01.179606 7273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: Bootstrap starting.
I20260810 21:04:01.180599 7273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: Neither blocks nor log segments found. Creating new log.
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:01.180956 7273 log.cc:824] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: Log is configured to *not* fsync() on all Append() calls
I20260810 21:04:01.181790 7273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: No bootstrap required, opened a new log
I20260810 21:04:01.183761 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.176187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.14.200.61:33333)
0810 21:04:01.176230 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.176232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.176290 (+ 58us) server_negotiation.cc:407] Connection header received
0810 21:04:01.176362 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.176363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.176371 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.176411 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.176997 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.177667 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.178523 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.178800 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.178875 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.181162 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.181164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.181164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.181172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.182683 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.182989 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.182990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.182990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.183062 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.183240 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.183241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.183242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.183376 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.183538 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.183579 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.183661 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:04:01.184000 7273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.184175 7273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29df537b84ad42208e95c1717d5c50d0, State: Initialized, Role: FOLLOWER
I20260810 21:04:01.184267 7273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.184564 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.184633 7273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:01.184695 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.185319 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 21:04:01.186558 7294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:01.186615 7294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:01.186230 7070 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "573d619267bf4ac68a71354d3192fe0a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45696
I20260810 21:04:01.186859 7196 master_runner.cc:186] Error getting master registration for 127.14.200.62:41795: OK, instance_id { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" instance_seqno: 1786395840795953 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:04:01.188964 7196 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:04:01.198881 7188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 3
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:01.207206 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.199256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56344)
0810 21:04:01.199324 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.199336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.199388 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.199578 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.199579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.199771 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.199830 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.199831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.200711 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.200711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.201531 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.201533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.202027 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.202028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.202100 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.202652 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.202658 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.204264 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.206064 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.206065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.206065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.206286 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.206495 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.206496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.206496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.206563 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.206877 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.206878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.206923 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.206997 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.207095 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 21:04:01.207211 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.199329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.14.200.60:42017)
0810 21:04:01.199389 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.199392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.199403 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:04:01.199477 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.199478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.199491 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.199570 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.199899 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.200626 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.201647 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.201942 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.201992 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.204351 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.204353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.204354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.204362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.205978 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.206357 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.206358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.206359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.206421 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.206621 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.206622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.206622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.206811 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.206881 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.207040 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.207120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 21:04:01.209088 7188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: Bootstrap starting.
I20260810 21:04:01.209880 7188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: Neither blocks nor log segments found. Creating new log.
I20260810 21:04:01.210211 7188 log.cc:824] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: Log is configured to *not* fsync() on all Append() calls
I20260810 21:04:01.211104 7188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: No bootstrap required, opened a new log
I20260810 21:04:01.213205 7188 raft_consensus.cc:348] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.213358 7188 raft_consensus.cc:729] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62e4188cfae4d5c9c36cbc78c8c6707, State: Initialized, Role: FOLLOWER
I20260810 21:04:01.213445 7188 consensus_queue.cc:260] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.213805 7188 sys_catalog.cc:564] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:01.213788 7297 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.213896 7297 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 21:04:01.215713 7309 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:01.215817 7309 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:01.215921 7108 master_runner.cc:186] Error getting master registration for 127.14.200.62:41795: OK, instance_id { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" instance_seqno: 1786395840795953 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:04:01.218900 7108 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:04:01.237669 7100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } attempt: 3
Aug 10 21:04:01 dist-test-slave-1x4g krb5kdc[7008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:01.245291 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.238049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:42017 (local address 127.0.0.1:56354)
0810 21:04:01.238109 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:01.238122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:01.238171 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:01.238326 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:01.238327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:01.238503 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:01.238553 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.238555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.239449 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.239451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.239960 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:01.239962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.240412 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.240413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:01.240481 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.240960 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:01.240971 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:01.242400 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:01.244204 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.244205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.244205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.244381 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.244599 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:01.244600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:01.244600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:01.244660 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:01.244953 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:01.244953 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:01.245000 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 21:04:01.245084 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 21:04:01.245188 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 21:04:01.245291 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.238115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.14.200.60:42017)
0810 21:04:01.238156 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.238158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.238168 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:04:01.238252 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.238253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.238264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.238312 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.238637 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.239373 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.240047 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.240343 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.240415 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.242500 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.242501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.242501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.242509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.244114 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.244477 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.244478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.244478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.244540 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.244730 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.244731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.244731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.244901 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.244960 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.245126 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.245200 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:04:01.247306 7100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: Bootstrap starting.
I20260810 21:04:01.248063 7100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: Neither blocks nor log segments found. Creating new log.
I20260810 21:04:01.248380 7100 log.cc:824] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: Log is configured to *not* fsync() on all Append() calls
I20260810 21:04:01.249262 7100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: No bootstrap required, opened a new log
I20260810 21:04:01.250643 7100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.250779 7100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573d619267bf4ac68a71354d3192fe0a, State: Initialized, Role: FOLLOWER
I20260810 21:04:01.250862 7100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.251192 7100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:01.251223 7070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 21:04:01.251267 7070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:01.252103 7070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.252400 7070 leader_election.cc:288] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [CANDIDATE]: Term 1 election: Requested vote from peers d62e4188cfae4d5c9c36cbc78c8c6707 (127.14.200.61:33333), 29df537b84ad42208e95c1717d5c50d0 (127.14.200.60:42017)
I20260810 21:04:01.252722 7160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "573d619267bf4ac68a71354d3192fe0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707"
I20260810 21:04:01.252875 7160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:01.253890 7160 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 573d619267bf4ac68a71354d3192fe0a in term 1.
I20260810 21:04:01.253893 7243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "573d619267bf4ac68a71354d3192fe0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29df537b84ad42208e95c1717d5c50d0"
I20260810 21:04:01.254000 7243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:01.254148 7040 leader_election.cc:302] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 573d619267bf4ac68a71354d3192fe0a, d62e4188cfae4d5c9c36cbc78c8c6707; no voters:
I20260810 21:04:01.254938 7243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 573d619267bf4ac68a71354d3192fe0a in term 1.
I20260810 21:04:01.251166 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.255390 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.255519 7312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:04:01.255606 7312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [term 1 LEADER]: Becoming Leader. State: Replica: 573d619267bf4ac68a71354d3192fe0a, State: Running, Role: LEADER
I20260810 21:04:01.255810 7312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } }
I20260810 21:04:01.256847 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d619267bf4ac68a71354d3192fe0a. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.256922 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: This master's current role is: LEADER
W20260810 21:04:01.258279 7330 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:01.258350 7330 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:01.261297 7332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:04:01.261523 7332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:04:01.265476 7243 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [term 1 FOLLOWER]: Refusing update from remote peer 573d619267bf4ac68a71354d3192fe0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:04:01.265628 7160 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [term 1 FOLLOWER]: Refusing update from remote peer 573d619267bf4ac68a71354d3192fe0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:04:01.265786 7328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:04:01.265956 7312 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:04:01.267023 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d619267bf4ac68a71354d3192fe0a. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.267114 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.267285 7297 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d619267bf4ac68a71354d3192fe0a. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.267360 7297 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.268253 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.268334 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: This master's current role is: LEADER
I20260810 21:04:01.268700 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.268811 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.269140 7337 mvcc.cc:205] Tried to move back new op lower bound from 7317077365820784640 to 7317077365784776704. Current Snapshot: MvccSnapshot[applied={T|T < 7317077365820784640}]
I20260810 21:04:01.269932 7328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.270015 7328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a [sys.catalog]: This master's current role is: LEADER
I20260810 21:04:01.270423 7297 sys_catalog.cc:455] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "573d619267bf4ac68a71354d3192fe0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "573d619267bf4ac68a71354d3192fe0a" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 41795 } } peers { permanent_uuid: "d62e4188cfae4d5c9c36cbc78c8c6707" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 33333 } } peers { permanent_uuid: "29df537b84ad42208e95c1717d5c50d0" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 42017 } } }
I20260810 21:04:01.270514 7297 sys_catalog.cc:458] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:01.270742 7342 mvcc.cc:205] Tried to move back new op lower bound from 7317077365820784640 to 7317077365784776704. Current Snapshot: MvccSnapshot[applied={T|T < 7317077365820784640}]
I20260810 21:04:01.271292 7332 catalog_manager.cc:1347] Generated new cluster ID: c19c5849d4784c0381e6ae7aaafa6a17
I20260810 21:04:01.271342 7332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:04:01.278815 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.265056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.14.200.61:33333)
0810 21:04:01.265095 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.265097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.268914 (+ 3817us) server_negotiation.cc:407] Connection header received
0810 21:04:01.269014 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.269015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.269025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.269057 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.269817 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.270566 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.274243 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.274532 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.274575 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.275945 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.275946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.275946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.275953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.277584 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.277977 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.277978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.277979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.278035 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.278246 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.278247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.278247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.278362 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.278439 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.278637 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.278734 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:04:01.282097 7332 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:04:01.282789 7332 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:04:01.284868 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.257071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.14.200.62:41795)
0810 21:04:01.257113 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.257115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.265367 (+ 8252us) server_negotiation.cc:407] Connection header received
0810 21:04:01.265471 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.265472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.265482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.265511 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.266343 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.267101 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.269188 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.269513 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.269563 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.281919 (+ 12356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.281921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.281921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.281929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.283592 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.284008 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.284009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.284010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.284064 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.284250 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.284251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.284251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.284368 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.284446 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.284661 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.284765 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:04:01.285420 7340 mvcc.cc:205] Tried to move back new op lower bound from 7317077365820784640 to 7317077365784776704. Current Snapshot: MvccSnapshot[applied={T|T < 7317077365820784640}]
I20260810 21:04:01.288483 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:01.261150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.14.200.60:42017)
0810 21:04:01.261191 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.261193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.268236 (+ 7043us) server_negotiation.cc:407] Connection header received
0810 21:04:01.268325 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.268326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.268335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.268363 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.270136 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.270811 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.273048 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:01.273344 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:01.273384 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:01.285584 (+ 12200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:01.285585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:01.285585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:01.285593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:01.287257 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.287652 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.287654 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.287654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.287718 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:01.287932 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:01.287933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:01.287933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:01.288069 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:01.288141 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:01.288325 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 21:04:01.288398 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:04:01.291651 7332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573d619267bf4ac68a71354d3192fe0a: Generated new TSK 0
I20260810 21:04:01.291811 7332 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:04:01.321548 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.320479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.14.200.61:33333)
0810 21:04:01.320525 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.320528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.320886 (+ 358us) server_negotiation.cc:407] Connection header received
0810 21:04:01.320996 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.320998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.321010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.321041 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.321479 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53502: BlockingRecv error: recv got EOF from 127.0.0.1:53502 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 21:04:01.322765 7102 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.321918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45716 (local address 127.14.200.62:41795)
0810 21:04:01.321978 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.321980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.322237 (+ 257us) server_negotiation.cc:407] Connection header received
0810 21:04:01.322334 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.322335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.322346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.322376 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.322703 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45716: BlockingRecv error: recv got EOF from 127.0.0.1:45716 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260810 21:04:01.323513 7277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.322125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.14.200.60:42017)
0810 21:04:01.322164 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.322166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.323017 (+ 851us) server_negotiation.cc:407] Connection header received
0810 21:04:01.323112 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.323113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.323123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.323155 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.323452 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56372: BlockingRecv error: recv got EOF from 127.0.0.1:56372 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 21:04:01.325719 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.324915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.14.200.61:33333)
0810 21:04:01.324950 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.324952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.325212 (+ 260us) server_negotiation.cc:407] Connection header received
0810 21:04:01.325318 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.325319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.325335 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.325364 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.325675 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53512: BlockingRecv error: recv got EOF from 127.0.0.1:53512 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 21:04:01.328668 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.327902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.14.200.61:33333)
0810 21:04:01.327934 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.327936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.328178 (+ 242us) server_negotiation.cc:407] Connection header received
0810 21:04:01.328269 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.328270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.328279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.328307 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.328616 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53516: BlockingRecv error: recv got EOF from 127.0.0.1:53516 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 21:04:01.332377 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.331684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.14.200.61:33333)
0810 21:04:01.331716 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.331718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.331911 (+ 193us) server_negotiation.cc:407] Connection header received
0810 21:04:01.332006 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.332007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.332016 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.332044 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.332338 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53518: BlockingRecv error: recv got EOF from 127.0.0.1:53518 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 21:04:01.338148 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.337400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.14.200.61:33333)
0810 21:04:01.337451 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.337453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.337682 (+ 229us) server_negotiation.cc:407] Connection header received
0810 21:04:01.337773 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.337774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.337783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.337811 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.338108 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53528: BlockingRecv error: recv got EOF from 127.0.0.1:53528 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 21:04:01.348116 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.347232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.14.200.61:33333)
0810 21:04:01.347273 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.347274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.347560 (+ 286us) server_negotiation.cc:407] Connection header received
0810 21:04:01.347665 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.347666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.347678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.347707 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.348054 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53538: BlockingRecv error: recv got EOF from 127.0.0.1:53538 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 21:04:01.366600 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.365505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.14.200.61:33333)
0810 21:04:01.365571 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.365573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.365940 (+ 367us) server_negotiation.cc:407] Connection header received
0810 21:04:01.366080 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.366081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.366095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.366128 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.366540 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53548: BlockingRecv error: recv got EOF from 127.0.0.1:53548 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 21:04:01.400827 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.399968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.14.200.61:33333)
0810 21:04:01.400032 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.400035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.400205 (+ 170us) server_negotiation.cc:407] Connection header received
0810 21:04:01.400266 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.400267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.400282 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.400351 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.400740 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53550: BlockingRecv error: recv got EOF from 127.0.0.1:53550 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 21:04:01.466562 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.465831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.14.200.61:33333)
0810 21:04:01.465928 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.465930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.466004 (+ 74us) server_negotiation.cc:407] Connection header received
0810 21:04:01.466071 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.466072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.466082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.466123 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.466472 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53562: BlockingRecv error: recv got EOF from 127.0.0.1:53562 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 21:04:01.596405 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.595718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.14.200.61:33333)
0810 21:04:01.595795 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.595797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.595878 (+ 81us) server_negotiation.cc:407] Connection header received
0810 21:04:01.595946 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.595947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.595957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.596010 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.596345 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53572: BlockingRecv error: recv got EOF from 127.0.0.1:53572 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 21:04:01.854243 7192 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:01.853640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.14.200.61:33333)
0810 21:04:01.853691 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:01.853693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:01.853705 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:04:01.853779 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:01.853781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:01.853791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:01.853828 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:01.854164 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53584: BlockingRecv error: recv got EOF from 127.0.0.1:53584 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:04:02.187064 7294 catalog_manager.cc:1259] Loaded cluster ID: c19c5849d4784c0381e6ae7aaafa6a17
I20260810 21:04:02.187264 7294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: loading cluster ID for follower catalog manager: success
I20260810 21:04:02.188102 7294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: acquiring CA information for follower catalog manager: success
I20260810 21:04:02.188408 7294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29df537b84ad42208e95c1717d5c50d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:04:02.216275 7309 catalog_manager.cc:1259] Loaded cluster ID: c19c5849d4784c0381e6ae7aaafa6a17
I20260810 21:04:02.216375 7309 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: loading cluster ID for follower catalog manager: success
I20260810 21:04:02.217293 7309 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: acquiring CA information for follower catalog manager: success
I20260810 21:04:02.217554 7309 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d62e4188cfae4d5c9c36cbc78c8c6707: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:04:02.369869 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:02.367492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.14.200.61:33333)
0810 21:04:02.367606 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:02.367608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:02.367621 (+ 13us) server_negotiation.cc:407] Connection header received
0810 21:04:02.367685 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:02.367685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:02.367735 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:02.367823 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:02.367962 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.368423 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.369201 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.369391 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.369445 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:02.369583 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:02.369693 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:02.369750 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:02.369783 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 21:04:02.369800 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260810 21:04:02.374188 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:02.371622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.14.200.60:42017)
0810 21:04:02.371723 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:02.371724 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:02.371731 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:04:02.371789 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:02.371789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:02.371822 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:02.371880 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:02.371986 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.372617 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.373305 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.373517 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.373603 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:02.373788 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:02.373908 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:02.373997 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:02.374086 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 21:04:02.374113 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":62,"threads_started":1}
I20260810 21:04:02.379158 7420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:02.375839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.14.200.61:33333)
0810 21:04:02.375877 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:02.375880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:02.376119 (+ 239us) server_negotiation.cc:407] Connection header received
0810 21:04:02.376160 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:02.376161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:02.376172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:02.376221 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:02.376384 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.377075 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378272 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.378593 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378657 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:02.378806 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:02.378843 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:02.378900 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:02.378947 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 21:04:02.378969 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:04:02.379246 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:02.375897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.14.200.62:41795)
0810 21:04:02.376011 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:02.376013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:02.376197 (+ 184us) server_negotiation.cc:407] Connection header received
0810 21:04:02.376281 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:02.376282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:02.376334 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:02.376400 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:02.376580 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.377270 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378372 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.378658 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378722 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:02.378905 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:02.379004 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:02.379082 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:02.379128 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 21:04:02.379153 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260810 21:04:02.379374 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:02.376054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.14.200.60:42017)
0810 21:04:02.376104 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:02.376106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:02.376252 (+ 146us) server_negotiation.cc:407] Connection header received
0810 21:04:02.376326 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:02.376327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:02.376338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:02.376383 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:02.376769 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.377440 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378505 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:02.378790 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:02.378862 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:02.379081 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:02.379116 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:02.379145 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:02.379256 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 21:04:02.379279 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 21:04:02.380913 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7021
I20260810 21:04:02.390749 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7108
I20260810 21:04:02.397054 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7196
2026-08-10T21:04:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3001 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 21:04:02 dist-test-slave-1x4g krb5kdc[7442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 21:04:02 dist-test-slave-1x4g krb5kdc[7442](info): set up 2 sockets
Aug 10 21:04:02 dist-test-slave-1x4g krb5kdc[7442](info): commencing operation
krb5kdc: starting...
W20260810 21:04:03.464888 15136 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/test-user.keytab.
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T21:04:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T21:04:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.200.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
Entry for principal kudu/127.14.200.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab.
I20260810 21:04:03.583631 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:34309
--webserver_interface=127.14.200.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 21:04:03.665175 7458 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:03.665373 7458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:03.665391 7458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:03.665410 7458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:03.666539 7458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:03.666576 7458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:03.666590 7458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:03.666646 7458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:03.666672 7458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:03.668151 7458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.62.keytab
--principal=kudu/127.14.200.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.62:34309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.200.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:03.668408 7458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:03.668632 7458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:03.673322 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:03.673398 7458 server_base.cc:1034] running on GCE node
W20260810 21:04:03.673337 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:03.673348 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:03.673825 7458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:03.674095 7458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:03.675267 7458 hybrid_clock.cc:648] HybridClock initialized: now 1786395843675246 us; error 40 us; skew 500 ppm
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:03.677608 7458 init.cc:377] Logged in from keytab as kudu/127.14.200.62@KRBTEST.COM (short username kudu)
I20260810 21:04:03.678045 7458 webserver.cc:460] Webserver started at http://127.14.200.62:36799/ using document root <none> and password file <none>
I20260810 21:04:03.678259 7458 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:03.678323 7458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:03.678467 7458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:03.679438 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data/instance:
uuid: "7555a01f776941c98deec1e0933c9700"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "9c3af9da936c62f23d5b8300ea690e11"
server_key_iv: "9ed9d7f9616c987c32909b31ab048c93"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.679791 7458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance:
uuid: "7555a01f776941c98deec1e0933c9700"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "9c3af9da936c62f23d5b8300ea690e11"
server_key_iv: "9ed9d7f9616c987c32909b31ab048c93"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.681192 7458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 21:04:03.681946 7478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:03.682137 7458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:03.682247 7458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal
uuid: "7555a01f776941c98deec1e0933c9700"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "9c3af9da936c62f23d5b8300ea690e11"
server_key_iv: "9ed9d7f9616c987c32909b31ab048c93"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.682320 7458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:03.689337 7458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:03.690990 7458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:03.691159 7458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:03.695118 7458 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.62:34309
I20260810 21:04:03.695165 7538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.62:34309 every 8 connection(s)
I20260810 21:04:03.695519 7458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/data/info.pb
I20260810 21:04:03.696504 7539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:03.698024 7539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:03.698421 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7458
I20260810 21:04:03.698523 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-0/wal/instance
I20260810 21:04:03.698740 15136 external_mini_cluster.cc:1428] Setting key b610d3f0b94648d81771a92ac043243b
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.200.61@KRBTEST.COM; defaulting to no policy
I20260810 21:04:03.708246 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.698791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34309 (local address 127.0.0.1:42390)
0810 21:04:03.698981 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.699024 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:03.699480 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:03.699804 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:03.699807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:03.700047 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:03.700200 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.700204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.701423 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.701424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.702280 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.702282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.703016 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.703017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.703070 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.703593 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:03.703601 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:03.704986 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:03.706677 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.706683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.706684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.706919 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.707158 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.707158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.707159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.707247 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.707643 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:03.707649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:03.707718 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 21:04:03.707843 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 21:04:03.708068 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":95,"thread_start_us":93,"threads_started":1}
I20260810 21:04:03.708245 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.698819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.14.200.62:34309)
0810 21:04:03.699056 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:03.699061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:03.699101 (+ 40us) server_negotiation.cc:407] Connection header received
0810 21:04:03.699634 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:03.699637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:03.699659 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:03.699757 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:03.700302 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.701319 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.702371 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.702896 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.703041 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.705070 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:03.705077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:03.705078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:03.705084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:03.706577 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.707018 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.707021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.707021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.707091 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.707311 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.707312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.707312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.707572 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:03.707662 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:03.707880 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 21:04:03.708007 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260810 21:04:03.709086 7539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:03.709403 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.709294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42468)
0810 21:04:03.709341 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.709382 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 21:04:03.709543 7482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.61@KRBTEST.COM" created.
I20260810 21:04:03.710021 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.709890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42470)
0810 21:04:03.709936 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.709982 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 21:04:03.710188 7539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:34485: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
Entry for principal kudu/127.14.200.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab.
I20260810 21:04:03.723868 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:34485
--webserver_interface=127.14.200.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:04:03.759421 7539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } attempt: 1
I20260810 21:04:03.759934 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.759795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42472)
0810 21:04:03.759863 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.759905 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 21:04:03.760277 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.760184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42478)
0810 21:04:03.760228 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.760254 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 21:04:03.760362 7539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:34485: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
W20260810 21:04:03.804302 7547 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:03.804534 7547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:03.804584 7547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:03.804620 7547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:03.805812 7547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:03.805876 7547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:03.805904 7547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:03.805933 7547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:03.805958 7547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:03.807452 7547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.61.keytab
--principal=kudu/127.14.200.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.61:34485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.200.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:03.807709 7547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:03.807915 7547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:03.812701 7553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:03.812711 7556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:03.812713 7552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:03.812953 7547 server_base.cc:1034] running on GCE node
I20260810 21:04:03.813190 7547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:03.813488 7547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:03.814649 7547 hybrid_clock.cc:648] HybridClock initialized: now 1786395843814638 us; error 30 us; skew 500 ppm
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:03.817219 7547 init.cc:377] Logged in from keytab as kudu/127.14.200.61@KRBTEST.COM (short username kudu)
I20260810 21:04:03.817647 7547 webserver.cc:460] Webserver started at http://127.14.200.61:42755/ using document root <none> and password file <none>
I20260810 21:04:03.817898 7547 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:03.817962 7547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:03.818111 7547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:03.819161 7547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data/instance:
uuid: "26ca6959937e47f0ac81d09cb8d3d1c0"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "8f1fb7ad7f90ed58544cc848b02e0be4"
server_key_iv: "0125d05efac76266ab5cdbe1ade62db8"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.819519 7547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal/instance:
uuid: "26ca6959937e47f0ac81d09cb8d3d1c0"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "8f1fb7ad7f90ed58544cc848b02e0be4"
server_key_iv: "0125d05efac76266ab5cdbe1ade62db8"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.820858 7547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:04:03.821676 7565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:03.821962 7547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 21:04:03.822072 7547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal
uuid: "26ca6959937e47f0ac81d09cb8d3d1c0"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "8f1fb7ad7f90ed58544cc848b02e0be4"
server_key_iv: "0125d05efac76266ab5cdbe1ade62db8"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.822145 7547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:03.828490 7539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } attempt: 2
I20260810 21:04:03.828940 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.828807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42492)
0810 21:04:03.828857 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.828902 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 21:04:03.829293 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.829200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42498)
0810 21:04:03.829244 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.829272 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 21:04:03.829403 7539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.61:34485: Network error: Client connection negotiation failed: client connection to 127.14.200.61:34485: connect: Connection refused (error 111)
I20260810 21:04:03.841908 7547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:03.844210 7547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:03.844319 7547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:03.848549 7547 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.61:34485
I20260810 21:04:03.848582 7631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.61:34485 every 8 connection(s)
I20260810 21:04:03.848992 7547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/data/info.pb
I20260810 21:04:03.849952 7633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:03.851423 7633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:04:03.858492 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7547
I20260810 21:04:03.858568 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-1/wal/instance
I20260810 21:04:03.858803 15136 external_mini_cluster.cc:1428] Setting key a5359d8755bac7727e66e2629a0421ce
I20260810 21:04:03.860106 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.852099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.14.200.62:34309)
0810 21:04:03.852161 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:03.852163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:03.852362 (+ 199us) server_negotiation.cc:407] Connection header received
0810 21:04:03.852967 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:03.852969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:03.852982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:03.853047 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:03.853684 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.854161 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.855168 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.855366 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.855450 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.857525 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:03.857526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:03.857526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:03.857533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:03.858773 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.859164 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.859165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.859166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.859228 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.859470 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.859471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.859472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.859587 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:03.859663 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:03.859923 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 21:04:03.860005 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 21:04:03.860320 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.852088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34309 (local address 127.0.0.1:42394)
0810 21:04:03.852254 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.852287 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:03.852847 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:03.853118 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:03.853122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:03.853379 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:03.853574 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.853578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.854261 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.854263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.855065 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.855068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.855484 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.855486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.855604 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.856035 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:03.856043 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:03.857442 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:03.858861 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.858868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.858870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.859087 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.859300 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.859300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.859301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.859404 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.859655 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:03.859662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:03.859796 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 21:04:03.859902 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 21:04:03.860052 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260810 21:04:03.860812 7633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.200.60@KRBTEST.COM; defaulting to no policy
I20260810 21:04:03.868260 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.861100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42510)
0810 21:04:03.861144 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.861154 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:03.861209 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:03.861433 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:03.861434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:03.861570 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:03.861641 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.861642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.862446 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.862447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.862953 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.862954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.863441 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.863442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.863520 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.863925 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:03.863928 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:03.865397 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:03.867080 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.867081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.867082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.867309 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.867521 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.867522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.867523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.867590 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.867866 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:03.867867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:03.867926 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 21:04:03.868021 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 21:04:03.868135 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 21:04:03.868335 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.861144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.14.200.61:34485)
0810 21:04:03.861308 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:03.861312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:03.861331 (+ 19us) server_negotiation.cc:407] Connection header received
0810 21:04:03.861347 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:03.861348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:03.861357 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:03.861433 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:03.861710 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.862356 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.863041 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.863362 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.863445 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.865483 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:03.865490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:03.865491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:03.865497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:03.866984 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.867397 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.867400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.867400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.867451 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.867648 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.867648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.867649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.867791 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:03.867849 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:03.868056 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 21:04:03.868185 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260810 21:04:03.868942 7633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:03.869326 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.869207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33868)
0810 21:04:03.869266 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.869309 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 21:04:03.869454 7569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
I20260810 21:04:03.869860 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.869764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33870)
0810 21:04:03.869814 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.869838 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 21:04:03.869992 7633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37601: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.200.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
Entry for principal kudu/127.14.200.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab.
I20260810 21:04:03.885155 15136 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrRlFwH/build/release/bin/kudu
/tmp/dist-test-taskrRlFwH/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:37601
--webserver_interface=127.14.200.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.200.20:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 21:04:03.919200 7633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } attempt: 1
I20260810 21:04:03.919761 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.919612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33872)
0810 21:04:03.919697 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.919732 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 21:04:03.920121 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.920021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33888)
0810 21:04:03.920066 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.920090 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 21:04:03.920205 7633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37601: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
I20260810 21:04:03.920555 7539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } attempt: 3
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:03.927472 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.920866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42524)
0810 21:04:03.920920 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.920928 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:03.920997 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:03.921126 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:03.921127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:03.921293 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:03.921360 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.921361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.922016 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.922018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.922560 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:03.922561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.922962 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.922963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:03.923051 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.923513 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:03.923525 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:03.924833 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:03.926428 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.926429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.926429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.926603 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.926814 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:03.926815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:03.926815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:03.926874 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:03.927144 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:03.927145 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:03.927191 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 21:04:03.927278 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 21:04:03.927377 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 21:04:03.927474 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.920944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.14.200.61:34485)
0810 21:04:03.921005 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:03.921007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:03.921019 (+ 12us) server_negotiation.cc:407] Connection header received
0810 21:04:03.921048 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:03.921049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:03.921061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:03.921120 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:03.921446 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.921924 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.922647 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:03.922883 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:03.922963 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:03.924920 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:03.924922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:03.924922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:03.924929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:03.926342 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.926688 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.926689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.926689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.926747 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:03.926962 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:03.926963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:03.926963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:03.927077 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:03.927152 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:03.927302 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 21:04:03.927377 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 21:04:03.927911 7539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260810 21:04:03.928246 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.928154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33892)
0810 21:04:03.928199 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.928228 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 21:04:03.928570 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.928484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33896)
0810 21:04:03.928526 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.928548 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 21:04:03.928660 7539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37601: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
I20260810 21:04:03.959960 7539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } attempt: 1
I20260810 21:04:03.960419 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.960289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33898)
0810 21:04:03.960352 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.960391 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 21:04:03.960731 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.960638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33900)
0810 21:04:03.960682 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.960707 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 21:04:03.960866 7539 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37601: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
W20260810 21:04:03.968410 7641 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 21:04:03.968618 7641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 21:04:03.968636 7641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 21:04:03.968654 7641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 21:04:03.969756 7641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 21:04:03.969796 7641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 21:04:03.969810 7641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 21:04:03.969822 7641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 21:04:03.969870 7641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 21:04:03.971323 7641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.200.20:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.200.62:34309,127.14.200.61:34485,127.14.200.60:37601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/krb5kdc/kudu_127.14.200.60.keytab
--principal=kudu/127.14.200.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.200.60:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.200.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 20:11:33 UTC on ccycloud.quasar-zccrtj.root.comops.site
build id 4579040
I20260810 21:04:03.971577 7641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 21:04:03.971853 7641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 21:04:03.976701 7647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:03.976716 7651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 21:04:03.976716 7648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 21:04:03.976955 7641 server_base.cc:1034] running on GCE node
I20260810 21:04:03.977185 7641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 21:04:03.977499 7641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 21:04:03.978638 7641 hybrid_clock.cc:648] HybridClock initialized: now 1786395843978625 us; error 28 us; skew 500 ppm
Aug 10 21:04:03 dist-test-slave-1x4g krb5kdc[7442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 21:04:03.981099 7641 init.cc:377] Logged in from keytab as kudu/127.14.200.60@KRBTEST.COM (short username kudu)
I20260810 21:04:03.981496 7641 webserver.cc:460] Webserver started at http://127.14.200.60:42801/ using document root <none> and password file <none>
I20260810 21:04:03.981720 7641 fs_manager.cc:359] Metadata directory not provided
I20260810 21:04:03.981782 7641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 21:04:03.981923 7641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 21:04:03.982923 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data/instance:
uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "f01185c5786df58e0dbcc6134691126a"
server_key_iv: "e9574848cf383dc5d93b093acaa01702"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.983282 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal/instance:
uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "f01185c5786df58e0dbcc6134691126a"
server_key_iv: "e9574848cf383dc5d93b093acaa01702"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.984527 7641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 21:04:03.985419 7660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 21:04:03.985597 7641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 21:04:03.985687 7641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data,/tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal
uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad"
format_stamp: "Formatted at 2026-08-10 21:04:03 on dist-test-slave-1x4g"
server_key: "f01185c5786df58e0dbcc6134691126a"
server_key_iv: "e9574848cf383dc5d93b093acaa01702"
server_key_version: "encryptionkey@0"
I20260810 21:04:03.985770 7641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 21:04:03.988338 7633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } attempt: 2
I20260810 21:04:03.988708 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.988590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33902)
0810 21:04:03.988648 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.988683 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:04:03.989101 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:03.988966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33910)
0810 21:04:03.989030 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:03.989071 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 21:04:03.989235 7633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.200.60:37601: Network error: Client connection negotiation failed: client connection to 127.14.200.60:37601: connect: Connection refused (error 111)
I20260810 21:04:04.002717 7641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 21:04:04.004519 7641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 21:04:04.004678 7641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 21:04:04.008893 7641 rpc_server.cc:307] RPC server started. Bound to: 127.14.200.60:37601
I20260810 21:04:04.008937 7719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.200.60:37601 every 8 connection(s)
I20260810 21:04:04.009327 7641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/data/info.pb
I20260810 21:04:04.010370 7720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 21:04:04.010656 15136 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrRlFwH/build/release/bin/kudu as pid 7641
I20260810 21:04:04.010740 15136 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrRlFwH/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786395574307397-15136-0/minicluster-data/master-2/wal/instance
I20260810 21:04:04.011003 15136 external_mini_cluster.cc:1428] Setting key da3bafef5247dfa42796ec396cbb3840
I20260810 21:04:04.012496 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:04.021374 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.013223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.14.200.62:34309)
0810 21:04:04.013287 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.013289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.013438 (+ 149us) server_negotiation.cc:407] Connection header received
0810 21:04:04.013513 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.013514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.013526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.013581 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.014065 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.014778 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.015759 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.016039 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.016117 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.018213 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.018214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.018214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.018221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.019857 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.020260 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.020262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.020262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.020320 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.020603 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.020604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.020604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.020743 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.020866 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.021191 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.021275 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 21:04:04.021585 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.013180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.62:34309 (local address 127.0.0.1:42408)
0810 21:04:04.013336 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:04.013378 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:04.013457 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:04.013606 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:04.013609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:04.013879 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:04.013974 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.013979 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.014864 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.014864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.015692 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.015694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.016150 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.016152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.016223 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.016630 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:04.016637 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:04.018131 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:04.019944 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.019951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.019953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.020172 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.020393 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.020394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.020395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.020540 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.020862 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:04.020869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:04.021039 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 21:04:04.021140 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 21:04:04.021267 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260810 21:04:04.022044 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.011537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.14.200.60:37601)
0810 21:04:04.011737 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.011745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.011779 (+ 34us) server_negotiation.cc:407] Connection header received
0810 21:04:04.012268 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.012276 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.012334 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.012492 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.013267 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.014139 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.015083 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.015626 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.015730 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.018710 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.018719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.018719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.018726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.020374 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.020756 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.020759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.020760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.020844 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.021043 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.021044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.021044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.021212 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.021353 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.021834 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.021913 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260810 21:04:04.022159 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.62@KRBTEST.COM
I20260810 21:04:04.029528 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.022406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.61:34485 (local address 127.0.0.1:42536)
0810 21:04:04.022552 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:04.022562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:04.022607 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:04.022787 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:04.022788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:04.022953 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:04.023011 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.023012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.023718 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.023719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.024289 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.024291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.024636 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.024637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.024705 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.025094 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:04.025098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:04.026343 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:04.028469 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.028470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.028470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.028665 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.028877 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.028877 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.028878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.028946 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.029219 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:04.029219 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:04.029261 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 21:04:04.029333 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 21:04:04.029419 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 21:04:04.029541 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.022488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.14.200.61:34485)
0810 21:04:04.022552 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.022554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.022595 (+ 41us) server_negotiation.cc:407] Connection header received
0810 21:04:04.022697 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.022698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.022707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.022770 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.023080 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.023624 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.024373 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.024564 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.024649 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.026435 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.026436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.026437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.026443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.028388 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.028733 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.028735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.028735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.028807 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.029007 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.029008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.029008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.029154 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.029215 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.029349 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.029437 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 21:04:04.029753 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.022802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.14.200.62:34309)
0810 21:04:04.022837 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.022838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.022845 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:04:04.022902 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.022903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.022910 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.022950 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.023424 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.023965 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.024820 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.025141 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.025207 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.027199 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.027200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.027200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.027207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.028566 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.028968 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.028969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.028969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.029019 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.029211 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.029212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.029212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.029330 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.029469 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.029532 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.029699 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 21:04:04.029912 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.60@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.200.61@KRBTEST.COM
I20260810 21:04:04.036252 7539 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } attempt: 2
I20260810 21:04:04.036988 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.030248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33940)
0810 21:04:04.030291 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:04.030301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:04.030338 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:04.030443 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:04.030444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:04.030607 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:04.030647 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.030649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.031446 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.031446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.031991 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.031993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.032318 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.032319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.032377 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.032834 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:04.032840 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:04.034221 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:04.035872 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.035873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.035873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.036081 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.036305 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.036306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.036306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.036373 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.036677 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:04.036678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:04.036714 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 21:04:04.036822 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 21:04:04.036910 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 21:04:04.037252 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.030286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.14.200.60:37601)
0810 21:04:04.030321 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.030323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.030330 (+ 7us) server_negotiation.cc:407] Connection header received
0810 21:04:04.030395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.030396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.030403 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.030438 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.030714 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.031376 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.032066 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.032252 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.032303 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.034305 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.034306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.034307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.034312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.035795 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.036169 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.036170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.036170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.036227 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.036447 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.036448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.036448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.036580 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.036655 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.037094 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.037160 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 21:04:04.038224 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.031086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.14.200.61:34485)
0810 21:04:04.031129 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.031131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.031179 (+ 48us) server_negotiation.cc:407] Connection header received
0810 21:04:04.031245 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.031245 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.031252 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.031289 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.031756 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.032326 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.033041 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.033351 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.033417 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.035408 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.035410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.035410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.035417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.036923 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.037396 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.037397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.037397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.037453 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.037703 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.037704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.037704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.037837 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.038022 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.038068 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.038149 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 21:04:04.039312 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: Bootstrap starting.
I20260810 21:04:04.039474 15136 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 21:04:04.040104 7720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: Neither blocks nor log segments found. Creating new log.
I20260810 21:04:04.040387 7720 log.cc:824] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: Log is configured to *not* fsync() on all Append() calls
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.62@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:04.041296 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: No bootstrap required, opened a new log
I20260810 21:04:04.040437 7507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7555a01f776941c98deec1e0933c9700"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42410
I20260810 21:04:04.043228 7720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.043392 7720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee4bbc2f24b4f79aa4a00579cec20ad, State: Initialized, Role: FOLLOWER
I20260810 21:04:04.043486 7720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.043823 7720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:04.044312 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.036506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33956)
0810 21:04:04.036554 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:04.036564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:04.036613 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:04.037040 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:04.037041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:04.037217 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:04.037256 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.037258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.038537 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.038538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.039256 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.039259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.039691 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.039692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.039751 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.040152 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:04.040157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:04.041478 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:04.043105 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.043107 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.043107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.043327 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.043555 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.043556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.043556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.043643 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.043959 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:04.043961 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:04.044014 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 21:04:04.044099 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 21:04:04.044185 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 21:04:04.044569 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.036686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.14.200.60:37601)
0810 21:04:04.036882 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.036884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.036892 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:04:04.036913 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.036913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.036923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.037441 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.037478 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.038202 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.039339 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.039611 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.039691 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.041582 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.041583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.041583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.041589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.043006 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.043416 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.043417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.043417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.043478 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.043711 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.043712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.043712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.043875 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.043953 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.044161 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.044235 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":155,"threads_started":1}
I20260810 21:04:04.044691 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.044848 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.046118 7539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: Bootstrap starting.
W20260810 21:04:04.046126 7742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:04.046279 7742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:04.046783 7539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: Neither blocks nor log segments found. Creating new log.
I20260810 21:04:04.047096 7539 log.cc:824] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: Log is configured to *not* fsync() on all Append() calls
I20260810 21:04:04.047899 7539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: No bootstrap required, opened a new log
I20260810 21:04:04.046120 7641 master_runner.cc:186] Error getting master registration for 127.14.200.62:34309: OK, instance_id { permanent_uuid: "7555a01f776941c98deec1e0933c9700" instance_seqno: 1786395843694082 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:04:04.048117 7641 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:04:04.049347 7539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.049489 7539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7555a01f776941c98deec1e0933c9700, State: Initialized, Role: FOLLOWER
I20260810 21:04:04.049571 7539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.049870 7746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.049944 7507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 21:04:04.050042 7507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:04.050770 7507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.050985 7507 leader_election.cc:288] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [CANDIDATE]: Term 1 election: Requested vote from peers 26ca6959937e47f0ac81d09cb8d3d1c0 (127.14.200.61:34485), 4ee4bbc2f24b4f79aa4a00579cec20ad (127.14.200.60:37601)
I20260810 21:04:04.049880 7539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:04.052168 7689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7555a01f776941c98deec1e0933c9700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad"
I20260810 21:04:04.052284 7689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:04.053778 7689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7555a01f776941c98deec1e0933c9700 in term 1.
I20260810 21:04:04.054028 7481 leader_election.cc:302] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee4bbc2f24b4f79aa4a00579cec20ad, 7555a01f776941c98deec1e0933c9700; no voters:
I20260810 21:04:04.054299 7746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.054986 7765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 21:04:04.055078 7765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [term 1 LEADER]: Becoming Leader. State: Replica: 7555a01f776941c98deec1e0933c9700, State: Running, Role: LEADER
I20260810 21:04:04.055169 7765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
W20260810 21:04:04.057077 7767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:04.057137 7767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:04.057267 7746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7555a01f776941c98deec1e0933c9700. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.057334 7746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: This master's current role is: LEADER
I20260810 21:04:04.057696 7772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 21:04:04.057865 7772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 21:04:04.058804 7458 master_runner.cc:186] Error getting master registration for 127.14.200.61:34485: OK, instance_id { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" instance_seqno: 1786395843847373 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 21:04:04.061060 7458 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 21:04:04.065189 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.052556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.14.200.62:34309)
0810 21:04:04.052597 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.052599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.054660 (+ 2061us) server_negotiation.cc:407] Connection header received
0810 21:04:04.054750 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.054751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.054763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.054810 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.056205 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.056970 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.058043 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.058300 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.058341 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.061022 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.061024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.061024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.061032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.063153 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.064080 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.064082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.064093 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.064156 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.064486 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.064487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.064487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.064628 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.064712 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.064999 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.065088 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 21:04:04.067543 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.054101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.14.200.60:37601)
0810 21:04:04.054140 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.054142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.054860 (+ 718us) server_negotiation.cc:407] Connection header received
0810 21:04:04.055114 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.055115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.055126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.055157 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.056163 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.057068 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.061585 (+ 4517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.061897 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.061949 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.064574 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.064575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.064575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.064581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.066256 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.066645 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.066647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.066647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.066704 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.066891 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.066892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.066892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.067049 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.067138 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.067353 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.067454 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:04:04.069507 7689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [term 1 FOLLOWER]: Refusing update from remote peer 7555a01f776941c98deec1e0933c9700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:04:04.069900 7765 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 21:04:04.071224 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7555a01f776941c98deec1e0933c9700. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.071308 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.072378 7765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.072490 7765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: This master's current role is: LEADER
I20260810 21:04:04.072945 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.073040 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.073122 7773 mvcc.cc:205] Tried to move back new op lower bound from 7317077377306202112 to 7317077377254064128. Current Snapshot: MvccSnapshot[applied={T|T < 7317077377306202112}]
I20260810 21:04:04.074397 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.054520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.14.200.61:34485)
0810 21:04:04.054563 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.054565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.060914 (+ 6349us) server_negotiation.cc:407] Connection header received
0810 21:04:04.061032 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.061034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.061045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.061093 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.061741 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.062425 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.063461 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.063767 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.063816 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.071096 (+ 7280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.071098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.071098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.071107 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.072752 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.073534 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.073535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.073536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.073611 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.073828 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.073829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.073829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.073984 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.074077 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.074206 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.074302 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 21:04:04.074801 7775 mvcc.cc:205] Tried to move back new op lower bound from 7317077377306202112 to 7317077377254064128. Current Snapshot: MvccSnapshot[applied={T|T < 7317077377306202112}]
I20260810 21:04:04.075611 7772 catalog_manager.cc:1347] Generated new cluster ID: 3f4d48fd0f984280bf6f72b3cf3e2b5a
I20260810 21:04:04.075671 7772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 21:04:04.080348 7633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } attempt: 3
Aug 10 21:04:04 dist-test-slave-1x4g krb5kdc[7442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786395843, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.200.61@KRBTEST.COM for kudu/127.14.200.60@KRBTEST.COM
I20260810 21:04:04.089196 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.080670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.200.60:37601 (local address 127.0.0.1:33982)
0810 21:04:04.080716 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 21:04:04.080726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 21:04:04.080762 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 21:04:04.080971 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 21:04:04.080972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 21:04:04.081168 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 21:04:04.081212 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.081214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.082665 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.082666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.083476 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 21:04:04.083479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.084024 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.084025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 21:04:04.084094 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.084521 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 21:04:04.084525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 21:04:04.086123 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 21:04:04.087940 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.087941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.087941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.088173 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.088415 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 21:04:04.088416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 21:04:04.088416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 21:04:04.088486 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 21:04:04.088892 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 21:04:04.088893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 21:04:04.088948 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 21:04:04.089005 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 21:04:04.089100 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 21:04:04.089744 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:04.080731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.14.200.60:37601)
0810 21:04:04.080767 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.080769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.080809 (+ 40us) server_negotiation.cc:407] Connection header received
0810 21:04:04.080866 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.080867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.080890 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.080933 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.081291 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.082009 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.083584 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:04.083898 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:04.083955 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:04.086226 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 21:04:04.086227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 21:04:04.086227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 21:04:04.086234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 21:04:04.087834 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.088274 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.088275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.088276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.088341 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 21:04:04.088617 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 21:04:04.088618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 21:04:04.088618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 21:04:04.088768 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 21:04:04.088842 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:04.089590 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 21:04:04.089662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 21:04:04.092062 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: Bootstrap starting.
I20260810 21:04:04.092808 7633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: Neither blocks nor log segments found. Creating new log.
I20260810 21:04:04.093180 7633 log.cc:824] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: Log is configured to *not* fsync() on all Append() calls
I20260810 21:04:04.094116 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: No bootstrap required, opened a new log
I20260810 21:04:04.096191 7633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.096359 7633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ca6959937e47f0ac81d09cb8d3d1c0, State: Initialized, Role: FOLLOWER
I20260810 21:04:04.096439 7633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } }
I20260810 21:04:04.097470 7633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 21:04:04.097533 7597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7555a01f776941c98deec1e0933c9700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0"
I20260810 21:04:04.097630 7597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 21:04:04.098382 7597 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7555a01f776941c98deec1e0933c9700 in term 1.
I20260810 21:04:04.098821 7596 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [term 1 FOLLOWER]: Refusing update from remote peer 7555a01f776941c98deec1e0933c9700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 21:04:04.099270 7746 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 21:04:04.101291 7794 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 21:04:04.101378 7794 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 21:04:04.097435 7781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.102108 7781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.102170 7781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7555a01f776941c98deec1e0933c9700. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.102212 7795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.102363 7795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.102226 7781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 21:04:04.105809 7796 mvcc.cc:205] Tried to move back new op lower bound from 7317077377306202112 to 7317077377254064128. Current Snapshot: MvccSnapshot[applied={T|T < 7317077377306202112}]
I20260810 21:04:04.106894 7765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7555a01f776941c98deec1e0933c9700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7555a01f776941c98deec1e0933c9700" member_type: VOTER last_known_addr { host: "127.14.200.62" port: 34309 } } peers { permanent_uuid: "26ca6959937e47f0ac81d09cb8d3d1c0" member_type: VOTER last_known_addr { host: "127.14.200.61" port: 34485 } } peers { permanent_uuid: "4ee4bbc2f24b4f79aa4a00579cec20ad" member_type: VOTER last_known_addr { host: "127.14.200.60" port: 37601 } } }
I20260810 21:04:04.107009 7765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700 [sys.catalog]: This master's current role is: LEADER
I20260810 21:04:04.108649 7772 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 21:04:04.110370 7772 catalog_manager.cc:1504] Loading token signing keys...
I20260810 21:04:04.122363 7772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7555a01f776941c98deec1e0933c9700: Generated new TSK 0
I20260810 21:04:04.122587 7772 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 21:04:04.149812 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.148369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.14.200.61:34485)
0810 21:04:04.148416 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.148418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.149109 (+ 691us) server_negotiation.cc:407] Connection header received
0810 21:04:04.149217 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.149219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.149231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.149261 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.149736 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42576: BlockingRecv error: recv got EOF from 127.0.0.1:42576 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 21:04:04.150610 7725 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.148779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.14.200.60:37601)
0810 21:04:04.148834 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.148836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.150109 (+ 1273us) server_negotiation.cc:407] Connection header received
0810 21:04:04.150201 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.150202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.150212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.150241 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.150550 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33988: BlockingRecv error: recv got EOF from 127.0.0.1:33988 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 21:04:04.151677 7542 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.150988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.14.200.62:34309)
0810 21:04:04.151043 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.151045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.151169 (+ 124us) server_negotiation.cc:407] Connection header received
0810 21:04:04.151255 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.151257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.151266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.151294 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.151615 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42428: BlockingRecv error: recv got EOF from 127.0.0.1:42428 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 21:04:04.154265 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.153312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.14.200.61:34485)
0810 21:04:04.153362 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.153364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.153703 (+ 339us) server_negotiation.cc:407] Connection header received
0810 21:04:04.153818 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.153819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.153840 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.153870 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.154218 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42584: BlockingRecv error: recv got EOF from 127.0.0.1:42584 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 21:04:04.157351 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.156436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.14.200.61:34485)
0810 21:04:04.156469 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.156471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.156761 (+ 290us) server_negotiation.cc:407] Connection header received
0810 21:04:04.156941 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.156943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.156954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.156983 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.157307 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42594: BlockingRecv error: recv got EOF from 127.0.0.1:42594 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 21:04:04.161266 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.160414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42610 (local address 127.14.200.61:34485)
0810 21:04:04.160449 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.160451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.160701 (+ 250us) server_negotiation.cc:407] Connection header received
0810 21:04:04.160839 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.160840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.160852 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.160881 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.161226 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42610: BlockingRecv error: recv got EOF from 127.0.0.1:42610 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 21:04:04.167130 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.166318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.14.200.61:34485)
0810 21:04:04.166357 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.166358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.166604 (+ 246us) server_negotiation.cc:407] Connection header received
0810 21:04:04.166701 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.166703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.166713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.166742 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.167085 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42624: BlockingRecv error: recv got EOF from 127.0.0.1:42624 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 21:04:04.177407 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.176319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42630 (local address 127.14.200.61:34485)
0810 21:04:04.176361 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.176363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.176673 (+ 310us) server_negotiation.cc:407] Connection header received
0810 21:04:04.176892 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.176894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.176908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.176944 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.177354 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42630: BlockingRecv error: recv got EOF from 127.0.0.1:42630 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 21:04:04.195736 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.194728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.14.200.61:34485)
0810 21:04:04.194773 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.194775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.195098 (+ 323us) server_negotiation.cc:407] Connection header received
0810 21:04:04.195212 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.195214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.195227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.195258 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.195678 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42638: BlockingRecv error: recv got EOF from 127.0.0.1:42638 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 21:04:04.229950 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.229079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.14.200.61:34485)
0810 21:04:04.229130 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.229132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.229344 (+ 212us) server_negotiation.cc:407] Connection header received
0810 21:04:04.229442 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.229443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.229455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.229504 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.229853 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42652: BlockingRecv error: recv got EOF from 127.0.0.1:42652 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 21:04:04.295837 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.295103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42656 (local address 127.14.200.61:34485)
0810 21:04:04.295212 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.295214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.295254 (+ 40us) server_negotiation.cc:407] Connection header received
0810 21:04:04.295333 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.295334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.295345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.295398 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.295761 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42656: BlockingRecv error: recv got EOF from 127.0.0.1:42656 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 21:04:04.425840 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.425029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.14.200.61:34485)
0810 21:04:04.425106 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.425109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.425225 (+ 116us) server_negotiation.cc:407] Connection header received
0810 21:04:04.425320 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.425321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.425334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.425407 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.425767 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42670: BlockingRecv error: recv got EOF from 127.0.0.1:42670 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 21:04:04.683751 7638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 21:04:04.683132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.14.200.61:34485)
0810 21:04:04.683212 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:04.683213 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:04.683222 (+ 9us) server_negotiation.cc:407] Connection header received
0810 21:04:04.683250 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:04.683250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:04.683260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:04.683323 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 21:04:04.683669 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42686: BlockingRecv error: recv got EOF from 127.0.0.1:42686 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 21:04:05.046772 7742 catalog_manager.cc:1259] Loaded cluster ID: 3f4d48fd0f984280bf6f72b3cf3e2b5a
I20260810 21:04:05.046871 7742 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: loading cluster ID for follower catalog manager: success
I20260810 21:04:05.047677 7742 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: acquiring CA information for follower catalog manager: success
I20260810 21:04:05.048022 7742 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4ee4bbc2f24b4f79aa4a00579cec20ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:04:05.101876 7794 catalog_manager.cc:1259] Loaded cluster ID: 3f4d48fd0f984280bf6f72b3cf3e2b5a
I20260810 21:04:05.101967 7794 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: loading cluster ID for follower catalog manager: success
I20260810 21:04:05.102741 7794 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: acquiring CA information for follower catalog manager: success
I20260810 21:04:05.103004 7794 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26ca6959937e47f0ac81d09cb8d3d1c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 21:04:05.199932 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:05.197063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.14.200.61:34485)
0810 21:04:05.197225 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:05.197227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:05.197238 (+ 11us) server_negotiation.cc:407] Connection header received
0810 21:04:05.197315 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:05.197315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:05.197371 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:05.197445 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:05.197636 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.198131 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.199111 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.199353 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.199408 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:05.199576 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:05.199691 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:05.199751 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:05.199798 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 21:04:05.199817 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260810 21:04:05.204416 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:05.201864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.14.200.60:37601)
0810 21:04:05.201971 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:05.201973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:05.201983 (+ 10us) server_negotiation.cc:407] Connection header received
0810 21:04:05.202061 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:05.202062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:05.202109 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:05.202194 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:05.202289 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.202812 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.203570 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.203800 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.203867 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:05.204074 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:05.204174 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:05.204260 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:05.204298 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 21:04:05.204320 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":55,"threads_started":1}
I20260810 21:04:05.208765 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:05.206206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.14.200.60:37601)
0810 21:04:05.206339 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:05.206340 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:05.206380 (+ 40us) server_negotiation.cc:407] Connection header received
0810 21:04:05.206527 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:05.206528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:05.206538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:05.206588 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:05.206728 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.207172 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208057 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.208302 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208371 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:05.208514 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:05.208543 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:05.208567 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:05.208663 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 21:04:05.208689 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 21:04:05.209197 7875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:05.206158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.14.200.61:34485)
0810 21:04:05.206305 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:05.206307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:05.206340 (+ 33us) server_negotiation.cc:407] Connection header received
0810 21:04:05.206426 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:05.206427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:05.206436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:05.206512 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:05.206680 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.207392 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208203 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.208483 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208558 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:05.208698 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:05.208729 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:05.208754 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:05.209088 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 21:04:05.209111 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 21:04:05.209658 7890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 21:04:05.206204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.14.200.62:34309)
0810 21:04:05.206390 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 21:04:05.206409 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 21:04:05.206417 (+ 8us) server_negotiation.cc:407] Connection header received
0810 21:04:05.206504 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 21:04:05.206505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 21:04:05.206565 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 21:04:05.206642 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 21:04:05.206809 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.207540 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208441 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 21:04:05.208731 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 21:04:05.208820 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 21:04:05.209216 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 21:04:05.209422 (+ 206us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 21:04:05.209483 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 21:04:05.209553 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 21:04:05.209579 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":49,"threads_started":1}
I20260810 21:04:05.216933 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7458
I20260810 21:04:05.225382 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7547
I20260810 21:04:05.231379 15136 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrRlFwH/build/release/bin/kudu with pid 7641
2026-08-10T21:04:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2834 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5836 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (270923 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST